[2025-03-17 09:56:55.782703] INFO New syslog file info: [address: "192.168.1.62:2882", observer version: OceanBase_CE 4.3.5.0, revision: 100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5, sysname: Linux, os release: 5.10.134-18.an8.x86_64, machine: x86_64, tz GMT offset: 08:00] [2025-03-17 09:56:55.782528] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=19] [MEMORY] tenant_id= 1004 ctx_id= MEMSTORE_CTX_ID hold= 272,629,760 used= 270,499,840 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 270,499,840 used= 270,376,080 count= 130 avg_used= 2,079,816 block_cnt= 130 chunk_cnt= 130 mod=Memstore [MEMORY] hold= 270,499,840 used= 270,376,080 count= 130 avg_used= 2,079,816 mod=SUMMARY [2025-03-17 09:56:55.782538] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= TRANS_CTX_MGR_ID hold= 2,097,152 used= 1,458,176 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 1,458,176 used= 1,445,120 count= 2 avg_used= 722,560 block_cnt= 2 chunk_cnt= 1 mod=PartTranCtxMgr [MEMORY] hold= 1,458,176 used= 1,445,120 count= 2 avg_used= 722,560 mod=SUMMARY [2025-03-17 09:56:55.782550] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= PLAN_CACHE_CTX_ID hold= 652,214,272 used= 555,851,072 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 518,805,760 used= 466,472,173 count= 22,378 avg_used= 20,845 block_cnt= 22,378 chunk_cnt= 310 mod=SqlPhyPlan [MEMORY] hold= 25,143,552 used= 21,991,536 count= 2,015 avg_used= 10,913 block_cnt= 2,015 chunk_cnt= 298 mod=SqlPlanCache [MEMORY] hold= 7,652,992 used= 6,977,728 count= 3,517 avg_used= 1,984 block_cnt= 1,173 chunk_cnt= 296 mod=CreateContext [MEMORY] hold= 2,763,520 used= 2,698,240 count= 340 avg_used= 7,936 block_cnt= 340 chunk_cnt= 80 mod=OB_SQLSTAT_OBJ [MEMORY] hold= 1,483,264 used= 1,377,696 count= 156 avg_used= 8,831 block_cnt= 156 chunk_cnt= 1 mod=SqlPhyPlObj [MEMORY] hold= 1,984 used= 1,600 count= 2 avg_used= 800 block_cnt= 1 chunk_cnt= 1 mod=PlanCache [MEMORY] hold= 555,851,072 used= 499,518,973 count= 28,408 avg_used= 17,583 mod=SUMMARY [2025-03-17 09:56:55.782560] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= GLIBC hold= 29,360,128 used= 7,332,464 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 7,318,960 used= 6,007,142 count= 18,360 avg_used= 327 block_cnt= 1,028 chunk_cnt= 14 mod=PlJit [MEMORY] hold= 13,504 used= 13,344 count= 2 avg_used= 6,672 block_cnt= 2 chunk_cnt= 2 mod=PlCodeGen [MEMORY] hold= 7,332,464 used= 6,020,486 count= 18,362 avg_used= 327 mod=SUMMARY [2025-03-17 09:56:55.782568] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= CO_STACK hold= 119,537,664 used= 117,669,888 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 117,669,888 used= 117,451,008 count= 228 avg_used= 515,136 block_cnt= 228 chunk_cnt= 57 mod=CoStack [MEMORY] hold= 117,669,888 used= 117,451,008 count= 228 avg_used= 515,136 mod=SUMMARY [2025-03-17 09:56:55.782580] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= KVSTORE_CACHE_ID hold= 5,098,176,512 used= 5,098,176,512 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 5,098,176,512 used= 0 count= 2,431 avg_used= 0 block_cnt= 0 chunk_cnt= 0 mod=KvstorCacheMb [MEMORY] hold= 5,098,176,512 used= 0 count= 2,431 avg_used= 0 mod=SUMMARY [2025-03-17 09:56:55.782589] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= META_OBJ_CTX_ID hold= 75,497,472 used= 74,301,440 limit= 3,092,376,440 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 70,557,696 used= 62,770,587 count= 957 avg_used= 65,591 block_cnt= 957 chunk_cnt= 35 mod=L_TabletPool [MEMORY] hold= 3,342,336 used= 3,329,280 count= 51 avg_used= 65,280 block_cnt= 51 chunk_cnt= 2 mod=N_TabletPool [MEMORY] hold= 401,408 used= 400,064 count= 2 avg_used= 200,032 block_cnt= 2 chunk_cnt= 1 mod=PoolFreeList [MEMORY] hold= 74,301,440 used= 66,499,931 count= 1,010 avg_used= 65,841 mod=SUMMARY [2025-03-17 09:56:55.782597] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= TX_CALLBACK_CTX_ID hold= 6,291,456 used= 4,576,064 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 4,576,064 used= 4,467,968 count= 563 avg_used= 7,936 block_cnt= 563 chunk_cnt= 3 mod=MemtableCallbac [MEMORY] hold= 4,576,064 used= 4,467,968 count= 563 avg_used= 7,936 mod=SUMMARY [2025-03-17 09:56:55.782605] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= LOB_CTX_ID hold= 2,097,152 used= 16,256 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=LobPersist [MEMORY] hold= 8,128 used= 7,936 count= 1 avg_used= 7,936 block_cnt= 1 chunk_cnt= 1 mod=ExtInfoLog [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 mod=SUMMARY [2025-03-17 09:56:55.782620] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= TX_DATA_TABLE hold= 12,582,912 used= 6,185,920 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 4,243,328 used= 4,210,304 count= 6 avg_used= 701,717 block_cnt= 6 chunk_cnt= 4 mod=MEMTABLE_ARENA [MEMORY] hold= 1,926,336 used= 1,880,832 count= 237 avg_used= 7,936 block_cnt= 237 chunk_cnt= 6 mod=TX_DATA_SLICE [MEMORY] hold= 16,256 used= 15,872 count= 2 avg_used= 7,936 block_cnt= 2 chunk_cnt= 1 mod=TX_DATA_ARENA [MEMORY] hold= 6,185,920 used= 6,107,008 count= 245 avg_used= 24,926 mod=SUMMARY [2025-03-17 09:56:55.782630] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= MDS_DATA_ID hold= 4,194,304 used= 2,292,096 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 0 free_size= 0 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [MEMORY] hold= 2,292,096 used= 2,237,952 count= 282 avg_used= 7,936 block_cnt= 282 chunk_cnt= 2 mod=MdsTable [MEMORY] hold= 2,292,096 used= 2,237,952 count= 282 avg_used= 7,936 mod=SUMMARY [2025-03-17 09:56:55.782642] INFO [LIB] print_usage (ob_tenant_ctx_allocator.cpp:181) [117438][MemoryDump][T0][Y0-0000000000000000-0-0] [lt=4] [MEMORY] tenant_id= 1004 ctx_id= MERGE_RESERVE_CTX_ID hold= 33,554,432 used= 0 limit= 9,223,372,036,854,775,807 [MEMORY] idle_size= 33,554,432 free_size= 33,554,432 [MEMORY] wash_related_chunks= 0 washed_blocks= 0 washed_size= 0 [MEMORY] request_cached_chunk_cnt= 0 [2025-03-17 09:56:55.793324] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=18, avg_log_batch_cnt=1, total_group_log_size=10392, avg_group_log_size=577) [2025-03-17 09:56:55.812272] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=25] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:56:55.812309] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=30] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:56:55.812321] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=10] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=8417) [2025-03-17 09:56:55.826820] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859667503}], scn_array=[cnt:1, {val:1742176615792776000, v:0}], curr_size=588, accum_size=6167, time_cost=33486) [2025-03-17 09:56:55.830020] WDIAG [STORAGE.TRANS] inner_lock_for_read (ob_tx_data_functor.cpp:342) [117409][TimerWK2][T1][YB420A0B093E-00062FB95027E8C3-0-0] [lt=26][errcode=-6004] lock_for_read need retry(ret=-6004, tx_data={tx_id:{txid:14855878}, ref_cnt:1, state:"RUNNING", commit_version:{val:1742176615822497000, v:0}, start_scn:{val:1742176615822497000, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}, lock_for_read_arg_={mvcc_acc_ctx_:{type:1, abs_lock_timeout_ts:1742176645729785, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176615822497000, v:0}, tx_id:{txid:0}, scn:0}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:0}, tx_desc:NULL, tx_ctx:null, mem_ctx:null, tx_scn:0, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, data_trans_id_:{txid:14855878}, data_sql_sequence_:{branch:0, seq:3}, read_latest_:false, read_uncommitted_:false, scn_:{val:1742176615822497000, v:0}}, tx_cc_ctx={state:10, prepare_version:{val:1742176615822497000, v:0}}) [2025-03-17 09:56:55.846284] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=41] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=8) [2025-03-17 09:56:55.847874] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463472969}], scn_array=[cnt:1, {val:1742176615822497000, v:0}], curr_size=546, accum_size=546, time_cost=25118) [2025-03-17 09:56:55.854388] WDIAG [LIB.TIME] init (ob_timezone_info.cpp:39808) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=82][errcode=-4005] init twice(ret=-4005) [2025-03-17 09:56:55.854426] WDIAG [SHARE] init (ob_time_zone_info_manager.cpp:97) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=37][errcode=-4005] init tz info map failed(ret=-4005, tenant_id_=1004) [2025-03-17 09:56:55.854434] WDIAG [SHARE] fetch_time_zone_info (ob_time_zone_info_manager.cpp:171) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-4005] init failed(ret=-4005) [2025-03-17 09:56:55.854439] WDIAG [SERVER.OMT] operator() (ob_tenant_timezone_mgr.cpp:56) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4005] fail to update time zone info(ret=-4005) [2025-03-17 09:56:55.856278] WDIAG [LIB.TIME] init (ob_timezone_info.cpp:39808) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-4005] init twice(ret=-4005) [2025-03-17 09:56:55.856298] WDIAG [SHARE] init (ob_time_zone_info_manager.cpp:97) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4005] init tz info map failed(ret=-4005, tenant_id_=1006) [2025-03-17 09:56:55.856308] WDIAG [SHARE] fetch_time_zone_info (ob_time_zone_info_manager.cpp:171) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-4005] init failed(ret=-4005) [2025-03-17 09:56:55.856324] WDIAG [SERVER.OMT] operator() (ob_tenant_timezone_mgr.cpp:56) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-4005] fail to update time zone info(ret=-4005) [2025-03-17 09:56:55.856582] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae329470, extra_info="") [2025-03-17 09:56:55.856621] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:56:55.856819] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=14] QSched: |cfg: root:1,x,0 net_in:1,1249M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:288K/67:15051 disk:288K/67:15051 t1m2:72K/17:14121 t1m2g0:72K/17:14121 t1004m2:100K/24:15325 t1004m2g0:100K/24:15325 t1003m2:116K/26:15407 t1003m2g0:116K/26:15407 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:56:55.867057] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162410466}], scn_array=[cnt:1, {val:1742176615723621000, v:0}], curr_size=122, accum_size=957, time_cost=10384) [2025-03-17 09:56:55.867138] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872698321}], scn_array=[cnt:1, {val:1742176615723621001, v:0}], curr_size=122, accum_size=2202, time_cost=10501) [2025-03-17 09:56:55.877904] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=38][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:55.919180] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=36] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"192.168.1.62:2882", log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):44236, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):44236, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):258, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2025-03-17 09:56:55.920640] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=41][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:56:55.920666] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=25][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:56:55.926721] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859668091}], scn_array=[cnt:1, {val:1742176615792776001, v:0}], curr_size=122, accum_size=6289, time_cost=19683) [2025-03-17 09:56:55.930036] INFO [SHARE] adaptive_update_limit (ob_share_throttle_define.cpp:112) [3539253][T1004_TenantWea][T1003][YB420A0B093E-00062FB95AF675E6-0-0] [lt=29] adaptive update(Tenant ID=1003, Config Specify Resource Limit(MB)=819, TxShare Current Memory Limit(MB)=601, Holding Memory(MB)=59, Tenant Remain Memory(MB)=903, Usable Remain Memory(MB)=542, Last Update Limit Timestamp=1742176615929943, Is Updated=true) [2025-03-17 09:56:55.936792] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=7] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312795313521, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:55.936831] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=36][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:56:55.936843] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=11] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7aeb1ff7d0) [2025-03-17 09:56:55.936851] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=7] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312795313521, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:55.937020] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [117737][T1_Occam][T1][YB420A0B093E-00062FB7EFFFBFD0-0-0] [lt=34] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:56:55.937220] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=18] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2025-03-17 09:56:55.937243] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=20] start lib cache evict(tenant_id=1003, mem_hold=75497472, mem_limit=85899345, cache_obj_num=228, cache_node_num=228) [2025-03-17 09:56:55.937251] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=75497472, mem_limit=85899345, cache_obj_num=228, cache_node_num=228) [2025-03-17 09:56:55.937260] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=8] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:55.940008] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=6] Dumping All Cache Objs(alloc_obj_list.count()=228, alloc_obj_list=[{obj_id:33320, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184745}, {obj_id:35162, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191402}, {obj_id:35168, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202676}, {obj_id:35176, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202872}, {obj_id:35185, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194566}, {obj_id:35193, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190237}, {obj_id:35195, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193489}, {obj_id:35251, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179204}, {obj_id:35253, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:35255, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177976}, {obj_id:37271, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201056}, {obj_id:37386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233847}, {obj_id:38041, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237416}, {obj_id:38401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194846}, {obj_id:38952, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:38998, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39000, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39002, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39007, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193123}, {obj_id:39008, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185047}, {obj_id:39340, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182356}, {obj_id:39341, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181004}, {obj_id:39342, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179416}, {obj_id:39348, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196117}, {obj_id:39351, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201780}, {obj_id:39367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235254}, {obj_id:39369, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179880}, {obj_id:39402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199220}, {obj_id:39540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192539}, {obj_id:39697, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226000}, {obj_id:39909, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182976}, {obj_id:39919, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:39920, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180292}, {obj_id:39921, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187396}, {obj_id:39928, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:39930, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:39932, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:39937, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:39938, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:39939, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:39942, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:39945, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:39946, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:39947, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183646}, {obj_id:39949, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182873}, {obj_id:39950, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184278}, {obj_id:40084, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206365}, {obj_id:40107, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223790}, {obj_id:40108, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180350}, {obj_id:40109, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263305}, {obj_id:40110, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331790}, {obj_id:40111, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40129, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183490}, {obj_id:40158, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180900}, {obj_id:40159, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188646}, {obj_id:40160, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204918}, {obj_id:40193, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192613}, {obj_id:40275, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200576}, {obj_id:40276, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40277, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180004}, {obj_id:40280, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195104}, {obj_id:40281, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40282, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263647}, {obj_id:40284, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:40285, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182846}, {obj_id:40286, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181614}, {obj_id:40287, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:40288, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198336}, {obj_id:40289, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216980}, {obj_id:40290, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203406}, {obj_id:40291, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40292, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192722}, {obj_id:40293, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204622}, {obj_id:40294, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181130}, {obj_id:40295, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194163}, {obj_id:40296, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194023}, {obj_id:40297, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181347}, {obj_id:40298, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178480}, {obj_id:40299, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179672}, {obj_id:40300, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177992}, {obj_id:40301, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184955}, {obj_id:40302, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199077}, {obj_id:40303, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180472}, {obj_id:40304, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207773}, {obj_id:40305, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40306, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191932}, {obj_id:40307, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185119}, {obj_id:40308, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40309, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40310, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:40311, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203354}, {obj_id:40312, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178272}, {obj_id:40313, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40314, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182249}, {obj_id:40315, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248104}, {obj_id:40316, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40317, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:40318, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188408}, {obj_id:40324, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:40325, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:40327, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191558}, {obj_id:40330, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186614}, {obj_id:40331, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186470}, {obj_id:40332, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182421}, {obj_id:40334, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:40338, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183137}, {obj_id:40339, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179672}, {obj_id:40340, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180218}, {obj_id:40341, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207773}, {obj_id:40342, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:40343, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177800}, {obj_id:40344, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182821}, {obj_id:40345, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189868}, {obj_id:40346, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180883}, {obj_id:40347, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200944}, {obj_id:40348, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178491}, {obj_id:40349, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179802}, {obj_id:40350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183073}, {obj_id:40351, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:40353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:40354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:40356, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181614}, {obj_id:40357, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:40358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:40359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:40360, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178320}, {obj_id:40361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179563}, {obj_id:40362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178320}, {obj_id:40363, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:40364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:40368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193848}, {obj_id:40401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192816}, {obj_id:40402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208968}, {obj_id:40403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230424}, {obj_id:40406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300745}, {obj_id:40407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:40409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:40410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297044}, {obj_id:40411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40470, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276547}, {obj_id:40471, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251949}, {obj_id:40472, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219417}, {obj_id:40473, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280167}, {obj_id:40474, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224140}, {obj_id:40475, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221591}, {obj_id:40476, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:40477, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212536}, {obj_id:40478, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:40479, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189930}, {obj_id:40480, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183988}, {obj_id:40481, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204926}, {obj_id:40482, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235402}, {obj_id:40483, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182233}, {obj_id:40484, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179094}, {obj_id:40485, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199080}, {obj_id:40486, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179054}, {obj_id:40487, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179088}, {obj_id:40488, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203106}, {obj_id:40489, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226815}, {obj_id:40490, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215316}, {obj_id:40491, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206428}, {obj_id:40492, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40493, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205754}, {obj_id:40494, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206492}, {obj_id:40495, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264163}, {obj_id:40496, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194128}, {obj_id:40497, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40498, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221082}, {obj_id:40499, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190664}, {obj_id:40500, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191156}, {obj_id:40501, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40502, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:40503, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:40504, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:40505, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180726}, {obj_id:40506, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181131}, {obj_id:40507, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203744}, {obj_id:40508, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:295360}, {obj_id:40509, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203218}, {obj_id:40510, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:441835}, {obj_id:40511, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184744}, {obj_id:40512, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221820}, {obj_id:40513, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381057}, {obj_id:40514, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:376803}, {obj_id:40515, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274484}, {obj_id:40516, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250503}, {obj_id:40517, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200576}, {obj_id:40518, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185905}, {obj_id:40519, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269349}, {obj_id:40520, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:282801}, {obj_id:40521, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:253249}, {obj_id:40522, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268180}, {obj_id:40523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193179}, {obj_id:40524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200691}, {obj_id:40525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180248}, {obj_id:40526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:40527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182846}, {obj_id:40528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:40530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:40531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:40532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180844}, {obj_id:40533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:40534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181059}, {obj_id:40535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:40536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:40537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:40538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:40539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:40540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:40541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:40542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:40543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268481}, {obj_id:40544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246352}, {obj_id:40545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178818}, {obj_id:40546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264469}, {obj_id:40547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243566}, {obj_id:40548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202676}, {obj_id:40549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194233}, {obj_id:40550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180200}, {obj_id:40551, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192722}, {obj_id:40552, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272495}, {obj_id:40553, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249127}, {obj_id:40554, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192277}, {obj_id:40555, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203356}, {obj_id:40556, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191402}, {obj_id:40557, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184214}]) [2025-03-17 09:56:55.940381] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=372] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:55.943845] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=36] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:56:55.943891] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=31] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:56:55.956670] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:56:55.965551] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA581-0-0] [lt=0][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:55.968976] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463473515}], scn_array=[cnt:1, {val:1742176615942847000, v:0}], curr_size=587, accum_size=1133, time_cost=25073) [2025-03-17 09:56:55.975051] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175477][T1_L0_G5][T1][YB420A0B093F-00062FBA48857968-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1742176615974704000, start_id=1742176615974704000, end_id=1742176615974704001) [2025-03-17 09:56:55.975185] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=7] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40307215564, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:55.977369] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162410588}], scn_array=[cnt:1, {val:1742176615723621001, v:0}], curr_size=122, accum_size=1079, time_cost=20640) [2025-03-17 09:56:55.977962] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:55.982340] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [117741][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=41] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2025-03-17 09:56:55.992953] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:56:55.993705] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=59][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872698443}], scn_array=[cnt:1, {val:1742176615723621002, v:0}], curr_size=122, accum_size=2324, time_cost=37038) [2025-03-17 09:56:55.993807] INFO [LIB] stat (utility.h:1210) [3539272][T1004_ApplySrv2][T1004][YB420A0B093E-00062FB8DBD9BBB3-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=44, this=0x7f79ae39c280, extra_info="") [2025-03-17 09:56:55.993825] INFO [LIB] stat (utility.h:1210) [3539272][T1004_ApplySrv2][T1004][YB420A0B093E-00062FB8DBD9BBB3-0-0] [lt=17] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=20386, this=0x7f79ae39c4b0, extra_info="") [2025-03-17 09:56:55.993832] INFO [LIB] stat (utility.h:1210) [3539272][T1004_ApplySrv2][T1004][YB420A0B093E-00062FB8DBD9BBB3-0-0] [lt=8] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=0, this=0x7f79ae39c6e0, extra_info="") [2025-03-17 09:56:55.993840] INFO [LIB] stat (utility.h:1210) [3539272][T1004_ApplySrv2][T1004][YB420A0B093E-00062FB8DBD9BBB3-0-0] [lt=7] [APPLY STAT CB EXECUTE TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=-500, this=0x7f79ae39c910, extra_info="") [2025-03-17 09:56:55.993846] INFO [LIB] stat (utility.h:1210) [3539272][T1004_ApplySrv2][T1004][YB420A0B093E-00062FB8DBD9BBB3-0-0] [lt=6] [APPLY STAT CB TOTAL TIME](cur_stat_count=26, stat_interval=5000000, avg (count/cost)=19931, this=0x7f79ae39cb40, extra_info="") [2025-03-17 09:56:56.010510] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=31] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:56.010558] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=27] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:56.046405] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=7) [2025-03-17 09:56:56.056748] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:56:56.062104] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=1, size_used=65536, mem_used=124784640) [2025-03-17 09:56:56.062198] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=22] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:56.078051] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.080689] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=19, avg_log_submit_to_flush_time=25585, avg_log_submit_to_slide_time=25602) [2025-03-17 09:56:56.080756] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=4392, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:56.107103] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=21] tx gc loop thread is running(MTL_ID()=1003) [2025-03-17 09:56:56.107132] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=28] try gc retain ctx [2025-03-17 09:56:56.107141] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=8] try to retry start_working [2025-03-17 09:56:56.107175] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=14] finish update min start scn(max_decided_scn={val:1742176615992731000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176606086830001, v:0}, keep_alive_scn_:{val:1742176610989531001, v:0}, update_ts_:1742176616106865}) [2025-03-17 09:56:56.107786] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=16] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176626190689000, latest_log_ts:{val:1742176609587544001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:56:56.107886] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=65] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=9969) [2025-03-17 09:56:56.108515] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=32][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.108540] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.108550] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: 8ACD561A80300600, int: 1742176616107402}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: 8ACD561A80300600, int: 1742176616107402}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.108637] INFO [STORAGE] print_count (ob_query_iterator_factory.cpp:50) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=42] ObQueryIteratorFactory statistics(multi_scan_merge_alloc_count=0, multi_scan_merge_release_count=0, multi_get_merge_alloc_count=0, multi_get_merge_release_count=0, table_scan_alloc_count=0, table_scan_release_count=0, insert_dup_alloc_count=1750693, insert_dup_release_count=1750693, col_map_alloc_count=0, col_map_release_count=0) [2025-03-17 09:56:56.108664] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=18][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.109135] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=37] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2141}, to_seq_no={branch:0, seq:538}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415370} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.109203] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:678) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=48] rpc statistics(total_trans_msg_count=23, total_batch_msg_count=0) [2025-03-17 09:56:56.109218] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:152) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA52-0-0] [lt=14] transaction rpc statistics(total_rt=12493, total_process=163, avg_rt=76) [2025-03-17 09:56:56.110930] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA53-0-0] [lt=7] get number(ret=0, service_type_=0, range=1, base_id=1742176616110068000, start_id=1742176616110068000, end_id=1742176616110068001) [2025-03-17 09:56:56.111010] INFO [STORAGE.TRANS] process (ob_trans_rpc.cpp:149) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA53-0-0] [lt=21] transaction rpc statistics(total_rt=9679, total_process=116, avg_rt=82) [2025-03-17 09:56:56.116615] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:56:56.116658] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:6, fail_bw:24539, [delay/us]:prepare:4, schedule:28, submit:1, rt:13251, total:13285, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:56.116675] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4840, iops:11, bw:53167, [delay/us]:prepare:5, schedule:17, submit:19, rt:23762, total:23805) [2025-03-17 09:56:56.116685] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1, ref_cnt=4, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31964384, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:56.116731] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=42] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:6, avg_bw:24539, [delay/us]: prepare:4, schedule:28, submit:1, device:13251, total:13285) [2025-03-17 09:56:56.116745] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4841, avg_iops:11, avg_bw:53168, [delay/us]: prepare:5, schedule:17, submit:19, device:23762, total:23805) [2025-03-17 09:56:56.116758] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:56:56.116791] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32718, [delay/us]:prepare:1, schedule:25, submit:1, rt:9867, total:9895, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:56.116812] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4577, iops:17, bw:77707, [delay/us]:prepare:5, schedule:21, submit:78, rt:26003, total:26108) [2025-03-17 09:56:56.116824] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:56.116852] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=27] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32718, [delay/us]: prepare:1, schedule:25, submit:1, device:9867, total:9895) [2025-03-17 09:56:56.116869] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4578, avg_iops:17, avg_bw:77705, [delay/us]: prepare:5, schedule:21, submit:78, device:26003, total:26108) [2025-03-17 09:56:56.116881] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:56:56.116902] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8179, [delay/us]:prepare:4, schedule:18, submit:1, rt:16670, total:16693) [2025-03-17 09:56:56.116924] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4300, iops:20, bw:85884, [delay/us]:prepare:6, schedule:23, submit:2092, rt:18189, total:20311) [2025-03-17 09:56:56.116934] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:56.116961] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=24] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8179, [delay/us]: prepare:4, schedule:18, submit:1, device:16670, total:16693) [2025-03-17 09:56:56.116976] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4301, avg_iops:20, avg_bw:85883, [delay/us]: prepare:6, schedule:23, submit:2092, device:18189, total:20311) [2025-03-17 09:56:56.116988] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:56:56.117012] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4089, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:497, total:505) [2025-03-17 09:56:56.117034] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:56.117059] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4089, [delay/us]: prepare:0, schedule:0, submit:600000000, device:497, total:505) [2025-03-17 09:56:56.117079] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:56:56.117098] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:56.117110] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117121] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117129] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117136] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117143] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117162] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117167] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117176] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:56.117184] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117192] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117200] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117208] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:56.117218] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SERVER] net_in=137kB/s, net_out=91kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:56:56.118117] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=49] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:56:56.120501] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA54-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.120533] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA54-0-0] [lt=29][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.120547] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA54-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c6a0, local_datums:0x7f77cec0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c370, hex: 46FB561A80300600, int: 1742176616119110}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c378, hex: 46FB561A80300600, int: 1742176616119110}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.120676] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA54-0-0] [lt=57][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.120985] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA54-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3737}, to_seq_no={branch:0, seq:2678}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415371} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.127844] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=39] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=18717696) [2025-03-17 09:56:56.127904] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=14] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:56.139966] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:685) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=52] clear dtl interm result cost(us)(clear_cost=6252, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2025-03-17 09:56:56.152282] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1742176616151608000, start_id=1742176616151608000, end_id=1742176616151608001) [2025-03-17 09:56:56.152334] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=34] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:56:56.152361] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1742176616152346], gts:1742176616151608000, latest_srr:[mts=1742176616152346]}) [2025-03-17 09:56:56.152580] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:449) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=36] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:56.152599] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=19] try reload schema success(schema_version=1741661297536408, refresh_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_ret=0) [2025-03-17 09:56:56.152595] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:263) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F008849F-0-0] [lt=9] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:56.152615] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:288) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F008849F-0-0] [lt=18] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:56.152624] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1742176616152598], gts:1742176616151608000, latest_srr:[mts=1742176616152598]}) [2025-03-17 09:56:56.165630] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463475943}], scn_array=[cnt:1, {val:1742176616121776000, v:0}], curr_size=555, accum_size=3529, time_cost=43130) [2025-03-17 09:56:56.165762] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=6667, avg_group_log_size=512) [2025-03-17 09:56:56.166951] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:559) [117778][T1_HBService][T1][YB420A0B093E-00062FB7F27BCA18-0-0] [lt=17] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2025-03-17 09:56:56.167018] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859669502}], scn_array=[cnt:1, {val:1742176616129197000, v:0}], curr_size=587, accum_size=8165, time_cost=37024) [2025-03-17 09:56:56.167092] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=44] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13241, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:56:56.167114] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162410832}], scn_array=[cnt:1, {val:1742176616151608000, v:0}], curr_size=122, accum_size=1323, time_cost=10247) [2025-03-17 09:56:56.167131] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1323) [2025-03-17 09:56:56.167162] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=22, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:56:56.167128] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872698687}], scn_array=[cnt:1, {val:1742176616151608000, v:0}], curr_size=122, accum_size=2568, time_cost=10258) [2025-03-17 09:56:56.167174] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=45] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=2568) [2025-03-17 09:56:56.167174] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1323) [2025-03-17 09:56:56.167202] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:56:56.167215] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13343, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:56:56.167229] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=45, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:56:56.167242] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=2568) [2025-03-17 09:56:56.167264] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=2568, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:56.167777] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA56-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.167807] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA56-0-0] [lt=28][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.167821] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA56-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36d, hex: 62B3571A80300600, int: 1742176616166242}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c375, hex: 62B3571A80300600, int: 1742176616166242}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.167964] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA56-0-0] [lt=75][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.168273] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA56-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2437}, to_seq_no={branch:0, seq:1367}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415372} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.168314] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=32] PNIO [ratelimit] time: 1742176616168312, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007604, add_bytes: 0 [2025-03-17 09:56:56.174242] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=20] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:56:56.175517] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=7] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40311409868, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:56.178113] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.180932] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=48][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463476498}], scn_array=[cnt:1, {val:1742176616143117000, v:0}], curr_size=587, accum_size=4116, time_cost=15084) [2025-03-17 09:56:56.181452] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=28] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:56:56.183364] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:671) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=32] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=0, clean_start_pos=5400000, clean_num=200000) [2025-03-17 09:56:56.193718] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC4-0-0] [lt=27] [RS_MGR] force set rs list(master_rs="192.168.1.62:2882") [2025-03-17 09:56:56.193755] INFO [SERVER] handle_heartbeat (ob_service.cpp:3753) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC4-0-0] [lt=33] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"192.168.1.62:2882", server_id:4, rs_addr:"192.168.1.62:2882", rs_server_status:1, epoch_id:4}, hb_response={zone:"zone1", server:"192.168.1.62:2882", sql_port:2881, build_version:"4.3.5.0_100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5(Dec 31 2024 17:35:01)", start_service_time:1741316885024955, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=38) [2025-03-17 09:56:56.195648] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463477085}], scn_array=[cnt:1, {val:1742176616169759000, v:0}], curr_size=551, accum_size=4667, time_cost=14618) [2025-03-17 09:56:56.197845] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA58-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.197870] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA58-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.197880] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA58-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36d, hex: CF27581A80300600, int: 1742176616196047}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c375, hex: CF27581A80300600, int: 1742176616196047}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.197975] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA58-0-0] [lt=46][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.198250] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA58-0-0] [lt=28] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1399}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415373} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.199933] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117900][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA59-0-0] [lt=31] get number(ret=0, service_type_=0, range=1, base_id=1742176616199594000, start_id=1742176616199594000, end_id=1742176616199594001) [2025-03-17 09:56:56.223049] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859670089}], scn_array=[cnt:1, {val:1742176616192075000, v:0}], curr_size=588, accum_size=8753, time_cost=29817) [2025-03-17 09:56:56.228843] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463477636}], scn_array=[cnt:1, {val:1742176616199594000, v:0}], curr_size=549, accum_size=5216, time_cost=28818) [2025-03-17 09:56:56.231916] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117917][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5A-0-0] [lt=26][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.231943] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117917][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5A-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.231953] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117917][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5A-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f78c280c690, local_datums:0x7f78c280c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f78c280c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f78c280c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c36d, hex: 1AAC581A80300600, int: 1742176616229914}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c375, hex: 1AAC581A80300600, int: 1742176616229914}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f78c280c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.232044] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117917][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5A-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.232015] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2747) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF_DUMP](palf_id=1001, self="192.168.1.62:2882", [SlidingWindow]={palf_id:1001, self:"192.168.1.62:2882", lsn_allocator:{max_log_id:5827431, max_lsn:4162410954, max_scn:{val:1742176616151608000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:4162410954}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:4120467914}}, last_submit_lsn:{lsn:4162410832}, last_submit_end_lsn:{lsn:4162410954}, last_submit_log_id:5827431, last_submit_log_pid:1, max_flushed_lsn:{lsn:4162410832}, max_flushed_end_lsn:{lsn:4162410954}, max_flushed_log_pid:1, committed_end_lsn:{lsn:4162410954}, last_slide_log_id:5827431, last_slide_scn:{val:1742176616151608000, v:0}, last_slide_lsn:{lsn:4162410832}, last_slide_end_lsn:{lsn:4162410954}, last_slide_log_pid:1, last_slide_log_accum_checksum:3707689120, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f79c4a9a530}, [StateMgr]={this:0x7f79c4a9e830, self:"192.168.1.62:2882", palf_id:1001, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"192.168.1.62:2882", leader_epoch:0, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:5827432, is_changing_config_with_arb:false, reconfirm_start_time_us:1741595979373414, palf_role_change_cb:0x7f79c4aa52f0, allow_vote:true}, [ConfigMgr]={palf_id:1001, self:"192.168.1.62:2882", alive_paxos_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, alive_paxos_replica_num:1, log_ms_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, reconfig_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:1, config_seq:2}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f79c4a9bc70}, [ModeMgr]={palf_id:1001, self:"192.168.1.62:2882", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1741595966225063000, v:0}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1741595966225063000, v:0}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1741595966225063000, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1001, is_inited:true, min_block_max_scn:{val:18446744073709551615, v:3}, min_block_id:0, min_block_min_scn:{val:1741595966225063001, v:0}, base_lsn_for_block_gc:{lsn:4093390848}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:2, base_lsn:{lsn:4093390848}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1741595966225063000, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1741595966225063000, v:0}}}, log_meta_storage:{log_tail:{lsn:77824}, readable_log_tail:{lsn:77824}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4227600384}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1001}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1001/meta", dir_fd:270, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67026944, block_header_serialize_buf_:0x7f79c4aa0a38, flashback_version:0}, log_storage:{log_tail:{lsn:4162410954}, readable_log_tail:{lsn:4162410954}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4160495616}, min_scn:{val:1742176364319009003, v:0}, curr_block_id:62, palf_id:1001}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1001/log", dir_fd:274, min_block_id:0, max_block_id:63, curr_writable_block_id:62}, logical_block_size_:67104768, curr_block_writable_size_:65189430, block_header_serialize_buf_:0x7f79c4aa2738, flashback_version:0}, palf_epoch:2, last_purge_throttling_ts:1741595979372793, this:0x7f79c4a9ef70}, [Reconfirm]={palf_id:1001, self:"192.168.1.62:2882", state:"FINISHED", new_proposal_id:1, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:1, majority_max_log_server:"192.168.1.62:2882", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:0}, saved_end_lsn:{lsn:0}, last_submit_prepare_req_time_us:1741595979373670, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f79c4a9e9b0}) [2025-03-17 09:56:56.232172] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2751) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=153] [PALF_DUMP](palf_id=1001, self="192.168.1.62:2882", ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176616167170, last_flushed_end_lsn:{lsn:4162410954}}]) [2025-03-17 09:56:56.232186] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4325) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=13] current log_task status(palf_id=1001, self="192.168.1.62:2882", log_id=5827432, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176616167170, last_flushed_end_lsn:{lsn:4162410954}}]) [2025-03-17 09:56:56.232288] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5A-0-0] [lt=40] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2165}, to_seq_no={branch:0, seq:1062}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415374} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.236553] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=30] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c5be0, ref_cnt_:1, raw_buf_:0x7f7ac25a2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78620-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176616236106, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:404, callback_enqueue_used:2, callback_dequeue_used:21, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:56.236608] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=53][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.236648] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=39][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.236665] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=16][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.236673] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c5be0, ref_cnt_:1, raw_buf_:0x7f7ac25a2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78620-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176616236106, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:404, callback_enqueue_used:2, callback_dequeue_used:21, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:56.236708] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=24][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c5be0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78620-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176616236106, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:404, callback_enqueue_used:2, callback_dequeue_used:21, callback_finish_used:-1, end_used:1742176616236699}}, part_id:-1}) [2025-03-17 09:56:56.236725] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=7][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:56.237072] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=17][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.237092] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=19][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.237097] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaacc2510, src_buf_size=295) [2025-03-17 09:56:56.237102] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c64e0, ref_cnt_:1, raw_buf_:0x7f7aaacc1170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78620-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176616236776, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:264, callback_enqueue_used:2, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:56.237129] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=19][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c64e0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78620-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176616236776, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:264, callback_enqueue_used:2, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176616237122}}, part_id:-1}) [2025-03-17 09:56:56.237204] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78620-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade9850e0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222517519, assoc_session_id:3222517519, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176616235837, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176616235836, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a393d21f8, id:3222517519, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade9850e0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.246533] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=6) [2025-03-17 09:56:56.246560] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:56.246582] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:56.246912] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=31][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176614150155004, replayable_scn:1742176614150155004, readable_scn:1742176614150155004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176615681334000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:56.246942] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176614150155004, replayable_scn:1742176614150155004, readable_scn:1742176614150155004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176615681334000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:56.246956] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176614150155004, replayable_scn:1742176614150155004, readable_scn:1742176614150155004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176615681334000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:56.247899] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACF94-0-0] [lt=10][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:56.247942] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:56:56.247948] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:56:56.253372] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=14] ====== check clog disk timer task ====== [2025-03-17 09:56:56.253394] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=19] get_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:56:56.259884] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=69][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463478185}], scn_array=[cnt:1, {val:1742176616232637000, v:0}], curr_size=551, accum_size=5767, time_cost=26199) [2025-03-17 09:56:56.261674] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5C-0-0] [lt=8][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.261699] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5C-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.261710] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5C-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36d, hex: AE24591A80300600, int: 1742176616260782}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c375, hex: AE24591A80300600, int: 1742176616260782}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.261813] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5C-0-0] [lt=52][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.262091] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5C-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1034}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415375} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.263400] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5D-0-0] [lt=43] get number(ret=0, service_type_=0, range=1, base_id=1742176616262454000, start_id=1742176616262454000, end_id=1742176616262454001) [2025-03-17 09:56:56.269833] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=31] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:723, free_obj_hold_cnt:78, allocator used:3082248, allocator total:3329280}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:233, free_obj_hold_cnt:724, allocator used:62770587, allocator total:62770587}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():130944, full allocator used:1600, full allocator total:196480}, tablets_mem=66296347, tablets_mem_limit=3092376440, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:88, allocator used:231424, allocator total:261120}, pending_cnt=0, wait_gc_count=0, tablet count=956) [2025-03-17 09:56:56.278180] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=46][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.280572] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=54][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463478736}], scn_array=[cnt:1, {val:1742176616262454000, v:0}], curr_size=551, accum_size=6318, time_cost=17048) [2025-03-17 09:56:56.282464] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5E-0-0] [lt=6][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.282490] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5E-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.282490] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=22][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:301, tid:117786}]) [2025-03-17 09:56:56.282500] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5E-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: 4273591A80300600, int: 1742176616280898}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: 4273591A80300600, int: 1742176616280898}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.282605] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA5E-0-0] [lt=48][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.282938] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA5E-0-0] [lt=53] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1756}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415376} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.288752] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=15] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:56:56.289539] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=32] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:56:56.289626] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=35] inner load succ(latest_freeze_info={frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176607617607000, v:0}, frozen_statuses_.count():6, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741314277005166000, v:0}, schema_version:1741314275575456, data_version:17180067072}, {frozen_version:{val:1741370403684882000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741456800355942000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741543202134877000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}]}) [2025-03-17 09:56:56.291962] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=34] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1742176615992731000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=299225, in_cluster_service=true, cluster_version={val:1742176615992731000, v:0}, min_cluster_version={val:1742176615992731000, v:0}, max_cluster_version={val:1742176615992731000, v:0}, get_cluster_version_err=0, cluster_version_delta=299225, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=5762127, succ_cluster_heartbeat_count=8, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176264250519000, v:0}, local_cluster_delta=352041437, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:56:56.292210] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=29] PNIO [ratelimit] time: 1742176616292209, bytes: 2535004031658, bw: 0.275177 MB/s, add_ts: 1007615, add_bytes: 290741 [2025-03-17 09:56:56.294631] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463479287}], scn_array=[cnt:1, {val:1742176616283708000, v:0}], curr_size=551, accum_size=6869, time_cost=10090) [2025-03-17 09:56:56.296922] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA60-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.296951] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA60-0-0] [lt=27][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.296961] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA60-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: 76AD591A80300600, int: 1742176616295798}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: 76AD591A80300600, int: 1742176616295798}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.297064] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA60-0-0] [lt=41][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.297446] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA60-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1064}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415377} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.307273] INFO [LIB] stat (utility.h:1210) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:56:56.318456] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A76-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:56:56.318451] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463479838}], scn_array=[cnt:1, {val:1742176616298766000, v:0}], curr_size=553, accum_size=7422, time_cost=19399) [2025-03-17 09:56:56.318492] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A76-0-0] [lt=34] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:56:56.318518] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859670677}], scn_array=[cnt:1, {val:1742176616192075001, v:0}], curr_size=122, accum_size=8875, time_cost=11189) [2025-03-17 09:56:56.318551] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=32] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8875) [2025-03-17 09:56:56.318572] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=19, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:56:56.318590] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=37] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=7, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:56:56.318613] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=23] [PALF STAT LOG LIFE TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=27257, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:56:56.318627] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=18, avg_log_gen_to_freeze_time=2670, avg_log_gen_to_submit_time=2683, avg_log_submit_to_flush_time=24557, avg_log_submit_to_slide_time=24574) [2025-03-17 09:56:56.318645] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=36, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:56:56.318660] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8875, hit_cnt=18, read_cnt=18, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:56.320379] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add10a78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415378} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:815} write_epoch=1, prev=this=0x7f786a0baed0 trans_version={val:1742149865549115000, v:0} scn={val:1742149865549115000, v:0} tx_id={txid:14005241} prev=0x7f786a0bad20 next=(nil) modify_count=13 acc_checksum=3528955310 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4240} write_epoch=3, snapshot_version={val:1742176616318917000, v:0}, *this={this=0x7f7778feeb08 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f786a0baed0 latest_compact_node=0x7f7763e733e0 max_trans_version={val:1742176555705662000, v:0} max_trans_id=14415308 max_elr_trans_version={val:1742176555705662000, v:0} max_elr_trans_id=14415308 latest_compact_ts=1742148340327319 last_compact_cnt=11 total_trans_node_cnt=14 max_modify_scn={val:1742149865549115000, v:0} min_modify_scn={val:1742109728702608000, v:0}}) [2025-03-17 09:56:56.320465] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=85][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add10a78 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415378} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:815} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646219672, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646219672, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616318917000, v:0}, tx_id:{txid:14415378}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415378}, tx_desc:{this:0x7f7ae6bb59e0, tx_id:{txid:14415378}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616319986, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616319172, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:815}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.320565] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=98][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415378}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:815}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.320587] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=22][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant file data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415378} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.320623] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=34][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.320635] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: C5085A1A80300600, int: 1742176616319173}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: C5085A1A80300600, int: 1742176616319173}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 963A090100000000, int: 17382038}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0050610100000000, int: 23154688}]}) [2025-03-17 09:56:56.320743] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=66][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.321045] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA62-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1060}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415378} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.328123] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA64-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.328164] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA64-0-0] [lt=40][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.328175] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA64-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: D8255A1A80300600, int: 1742176616326616}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: D8255A1A80300600, int: 1742176616326616}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.328286] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA64-0-0] [lt=58][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.328578] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA64-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2913}, to_seq_no={branch:0, seq:1850}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415379} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.339360] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=17] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:56.347559] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=22] gc stale ls task succ [2025-03-17 09:56:56.372646] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463480944}], scn_array=[cnt:1, {val:1742176616329546000, v:0}], curr_size=553, accum_size=8528, time_cost=42244) [2025-03-17 09:56:56.372715] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859670799}], scn_array=[cnt:1, {val:1742176616329546000, v:0}], curr_size=588, accum_size=588, time_cost=42140) [2025-03-17 09:56:56.372719] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6cc0, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616340672, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:31994, callback_enqueue_used:3, callback_dequeue_used:24, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:56.372780] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=60][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.372806] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=25][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.372827] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.372835] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6cc0, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616340672, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:31994, callback_enqueue_used:3, callback_dequeue_used:24, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.372874] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6cc0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616340672, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:31994, callback_enqueue_used:3, callback_dequeue_used:24, callback_finish_used:-1, end_used:1742176616372870}}, part_id:-1}) [2025-03-17 09:56:56.372881] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=38][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616340672, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:31994, callback_enqueue_used:3, callback_dequeue_used:24, callback_finish_used:186, end_used:-32}}) [2025-03-17 09:56:56.372940] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=58][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:56.372954] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:56.374609] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA66-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.374630] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA66-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.374639] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA66-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: 67DC5A1A80300600, int: 1742176616373351}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: 67DC5A1A80300600, int: 1742176616373351}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.374731] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA66-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.375013] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA66-0-0] [lt=33] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2153}, to_seq_no={branch:0, seq:1078}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415380} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.377555] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.377589] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=33][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.377596] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79740b915a, src_buf_size=1611) [2025-03-17 09:56:56.377603] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c59a0, ref_cnt_:1, raw_buf_:0x7f79740b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616372966, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:4528, callback_enqueue_used:2, callback_dequeue_used:38, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.377637] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=28][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c59a0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616372966, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:4528, callback_enqueue_used:2, callback_dequeue_used:38, callback_finish_used:-1, end_used:1742176616377634}}, part_id:-1}) [2025-03-17 09:56:56.377653] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872698931}], scn_array=[cnt:1, {val:1742176616151608002, v:0}], curr_size=122, accum_size=244, time_cost=20678) [2025-03-17 09:56:56.377643] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176616372966, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:4528, callback_enqueue_used:2, callback_dequeue_used:38, callback_finish_used:104, end_used:-22}}) [2025-03-17 09:56:56.377695] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=51][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:56.377686] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162411076}], scn_array=[cnt:1, {val:1742176616151608002, v:0}], curr_size=122, accum_size=244, time_cost=20689) [2025-03-17 09:56:56.377710] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=12][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f79f0298080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.377733] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=22][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f79f0298080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.377752] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=16][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:56.377767] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=15][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:56.377789] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=22][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:56.377834] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=44][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:56.377843] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:56.377851] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:56.377862] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:56.377869] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646240500, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:56.377924] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=55][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646240500, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: C6575A1A80300600, int: 1742176616339398}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}}) [2025-03-17 09:56:56.377971] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=45][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: C6575A1A80300600, int: 1742176616339398}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:56.377996] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=24][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:56.378006] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:56.378013] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}, ret=-4016) [2025-03-17 09:56:56.378046] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7a22944e90, local_datums:0x7f7a22944ae0}, datums:[col_id=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: C6575A1A80300600, int: 1742176616339398}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944961, hex: C6575A1A80300600, int: 1742176616339398}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944969, hex: C6575A1A80300600, int: 1742176616339398}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944971, hex: 0700000000000000, int: 7}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944979, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7a22944981, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7a22944987, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 54, flag: 0, null: 0, ptr: 0x7f7a22944998, hex: 44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029, cstr: DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7a229449ce, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7a229449d7, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7a229449df, hex: 00D826AD68300600, int: 1742076000000000}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7a229449e7, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7a229449f8, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176646240500, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646240500, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616329546000, v:0}, tx_id:{txid:9634271}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634271}, tx_desc:{this:0x7f7acf29a560, tx_id:{txid:9634271}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518564, assoc_session_id:3222518564, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176616339163, active_ts:1742176616339163, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646339157, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460454739158, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176616329843}], exec_info_reap_ts:1742176616329842, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616329839, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.378128] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=80][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:56.378136] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: C6575A1A80300600, int: 1742176616339398}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: C6575A1A80300600, int: 1742176616339398}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.339398"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646240500, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:56.378200] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=63][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:56.378204] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=4][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:56.378213] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=4][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:56.378221] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:56.378234] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=10][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:56.378241] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:56.378240] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.378248] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:56.378256] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:56.378261] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=4][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176646340500, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388262, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf29a560, tx_id:{txid:9634271}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518564, assoc_session_id:3222518564, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176616339163, active_ts:1742176616339163, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646339157, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460454739158, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176616329843}], exec_info_reap_ts:1742176616329842, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616329839, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634271}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646240500, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646240500, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646240500, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:561, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:56.378765] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=504][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:56.378771] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=5][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:56.378779] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:56.378783] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=3][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:56.378787] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=4][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:56.378793] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=4][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:56.378803] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=9][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:56.378809] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:56.378816] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:56.378824] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:56.378830] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:56.378840] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:56.378850] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=9][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}) [2025-03-17 09:56:56.378863] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=12][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:56.378877] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=12][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:56.378900] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=14] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222518564, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf29a560}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:56.378928] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:45674}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634271} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:56.378949] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=17] end stmt(ret=0, tx_id=9634271, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf29a560, tx_id:{txid:9634271}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518564, assoc_session_id:3222518564, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176616339163, active_ts:1742176616339163, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646339157, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460454739158, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616329843}], exec_info_reap_ts:1742176616329842, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616329839, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222518564, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf29a560}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.378982] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=31][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:56.378989] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:56.378996] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:56.379010] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=6][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:56.379018] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:56.379028] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')) [2025-03-17 09:56:56.379046] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:56.379055] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW'), is_user_sql=false) [2025-03-17 09:56:56.379063] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616339398), usec_to_time(1742176616339398), 7, usec_to_time(1742176616339398), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW'), conn=0x7f7aed35bad0) [2025-03-17 09:56:56.379072] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:56.379176] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=10][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634271}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176646339157, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616340226}{self_ls_id:{id:1}, session_id:3222518564, associated_session_id:3222518564, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634271}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460454739158, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616340226, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634271} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.379236] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=48] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634271}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176646339157, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616340226}{self_ls_id:{id:1}, session_id:3222518564, associated_session_id:3222518564, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634271}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460454739158, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616340226, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634271} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.379275] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=33] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634271}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176616379172, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:56.379290] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=12] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf29a560, tx_id:{txid:9634271}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518564, assoc_session_id:3222518564, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176616339163, active_ts:1742176616339163, commit_ts:-1, finish_ts:1742176616378713, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646339157, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460454739158, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616329843}], exec_info_reap_ts:1742176616329842, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616329839, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:56.379377] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=23][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:56.379385] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=8][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:56.379393] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F0-0-0] [lt=7][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:56.383711] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40307215564, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:56.415382] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.415882] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.416091] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.416236] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.416308] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.416337] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.416569] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.431100] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=59][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463482084}], scn_array=[cnt:1, {val:1742176616375507000, v:0}], curr_size=551, accum_size=9666, time_cost=50999) [2025-03-17 09:56:56.432275] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=21] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:56.433843] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA68-0-0] [lt=8][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.433868] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA68-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.433878] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA68-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c6a0, local_datums:0x7f77cec0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c370, hex: 54C55B1A80300600, int: 1742176616432980}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c378, hex: 54C55B1A80300600, int: 1742176616432980}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.433978] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA68-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.434249] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA68-0-0] [lt=29] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3200}, to_seq_no={branch:0, seq:2134}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415381} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.435105] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:257) [117825][T1_ArcSender][T1][YB420A0B093E-00062FB7EE954125-0-0] [lt=30][errcode=-4018] try free send task failed(ret=-4018) [2025-03-17 09:56:56.443492] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:56.446652] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=27] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=5) [2025-03-17 09:56:56.455002] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859671387}], scn_array=[cnt:1, {val:1742176616392567000, v:0}], curr_size=588, accum_size=1176, time_cost=61665) [2025-03-17 09:56:56.455079] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.455140] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=59][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.455180] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=39][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.455192] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c60a8, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F1-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616444686, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:10351, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.455262] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=61][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c60a8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F1-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616444686, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:10351, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176616455256}}, part_id:-1}) [2025-03-17 09:56:56.455270] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=32][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616444686, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:10351, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:220, end_used:-39}}) [2025-03-17 09:56:56.455320] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=49][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:56.455334] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:56.462906] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463482635}], scn_array=[cnt:1, {val:1742176616434990000, v:0}], curr_size=557, accum_size=10223, time_cost=27364) [2025-03-17 09:56:56.462989] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=33][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.463026] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=36][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.463044] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=17][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79f029915a, src_buf_size=1611) [2025-03-17 09:56:56.463053] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6d08, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F1-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616455346, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:7606, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.463090] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6d08, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F1-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616455346, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:7606, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:1742176616463086}}, part_id:-1}) [2025-03-17 09:56:56.463098] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176616455346, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:7606, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:125, end_used:-25}}) [2025-03-17 09:56:56.463173] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=74][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:56.463196] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=19][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a22814080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.463224] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=27][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a22814080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.463249] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=21][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:56.463268] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=18][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:56.463296] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=28][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:56.463371] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=74][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:56.463390] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=18][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:56.463400] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:56.463413] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:56.463431] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=17][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646344556, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:56.463495] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=64][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646344556, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 84EE5B1A80300600, int: 1742176616443524}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}}) [2025-03-17 09:56:56.463541] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=44][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 84EE5B1A80300600, int: 1742176616443524}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:56.463565] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=24][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:56.463576] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:56.463583] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}, ret=-4016) [2025-03-17 09:56:56.463617] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7aa426ae10, local_datums:0x7f7aa426aa60}, datums:[col_id=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 84EE5B1A80300600, int: 1742176616443524}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8fd, hex: 84EE5B1A80300600, int: 1742176616443524}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a905, hex: 84EE5B1A80300600, int: 1742176616443524}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a90d, hex: 0800000000000000, int: 8}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a915, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7aa426a91d, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a923, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 28, flag: 0, null: 0, ptr: 0x7f7aa426a934, hex: 44424D535F53544154532E50555247455F5354415453284E554C4C29, cstr: DBMS_STATS.PURGE_STATS(NULL)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a950, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a959, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a961, hex: 1227F4050D300600, int: 1741682352858898}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a969, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a97a, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176646344556, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646344556, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616392567000, v:0}, tx_id:{txid:9634275}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634275}, tx_desc:{this:0x7f7acf37cf60, tx_id:{txid:9634275}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519830, assoc_session_id:3222519830, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176616443518, active_ts:1742176616443518, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646443512, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460558819884, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176616440314}], exec_info_reap_ts:1742176616440313, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616440310, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.463703] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=84][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:56.463712] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 84EE5B1A80300600, int: 1742176616443524}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 84EE5B1A80300600, int: 1742176616443524}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:56.443524"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176646344556, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:56.463761] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=48][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:56.463766] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=4][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:56.463776] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=3][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:56.463784] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:56.463800] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=10][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:56.463810] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:56.463820] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=10][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:56.463831] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:56.463841] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176646444556, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388270, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf37cf60, tx_id:{txid:9634275}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519830, assoc_session_id:3222519830, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176616443518, active_ts:1742176616443518, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646443512, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460558819884, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176616440314}], exec_info_reap_ts:1742176616440313, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616440310, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634275}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646344556, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646344556, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176646344556, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:56.464405] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=563][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:56.464412] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:56.464432] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=18][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:56.464438] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=5][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:56.464445] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:56.464451] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=5][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:56.464459] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:56.464463] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=3][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:56.464470] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:56.464478] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:56.464485] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=6][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:56.464493] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:56.464501] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}) [2025-03-17 09:56:56.464511] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=10][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:56.464522] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:56.464541] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222519830, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf37cf60}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:56.464578] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:23481}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634275} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:56.464621] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=39] end stmt(ret=0, tx_id=9634275, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf37cf60, tx_id:{txid:9634275}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519830, assoc_session_id:3222519830, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176616443518, active_ts:1742176616443518, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646443512, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460558819884, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616440314}], exec_info_reap_ts:1742176616440313, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616440310, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222519830, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf37cf60}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.464655] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6A-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.464672] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=49][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:56.464681] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:56.464686] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=4][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:56.464677] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6A-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.464698] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=5][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:56.464707] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:56.464717] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')) [2025-03-17 09:56:56.464737] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:56.464745] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), is_user_sql=false) [2025-03-17 09:56:56.464693] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6A-0-0] [lt=13][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c6a0, local_datums:0x7f77cec0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c370, hex: A83D5C1A80300600, int: 1742176616463784}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c378, hex: A83D5C1A80300600, int: 1742176616463784}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.464754] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176616443524), usec_to_time(1742176616443524), 8, usec_to_time(1742176616443524), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), conn=0x7f7b317c6a50) [2025-03-17 09:56:56.464763] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=8][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:56.464833] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6A-0-0] [lt=69][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.464840] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=4][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634275}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176646443512, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616443518}{self_ls_id:{id:1}, session_id:3222519830, associated_session_id:3222519830, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634275}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460558819884, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616444593, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634275} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.464901] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=50] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634275}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176646443512, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616443518}{self_ls_id:{id:1}, session_id:3222519830, associated_session_id:3222519830, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634275}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460558819884, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616444593, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634275} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.464953] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=49] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634275}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176616464836, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:56.464975] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=19] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf37cf60, tx_id:{txid:9634275}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519830, assoc_session_id:3222519830, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176616443518, active_ts:1742176616443518, commit_ts:-1, finish_ts:1742176616464784, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646443512, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460558819884, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616440314}], exec_info_reap_ts:1742176616440313, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616440310, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:56.465044] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:146) [3539202][T1004_TntTransf][T1004][YB420A0B093E-00062FB7FC2623F0-0-0] [lt=23] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=2) [2025-03-17 09:56:56.465077] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=36][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:56.465086] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=9][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:56.465094] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F1-0-0] [lt=7][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:56.465086] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA6A-0-0] [lt=33] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1071}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415382} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.465302] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=24] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}]) [2025-03-17 09:56:56.477737] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872699053}], scn_array=[cnt:1, {val:1742176616151608003, v:0}], curr_size=122, accum_size=366, time_cost=20696) [2025-03-17 09:56:56.477774] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162411198}], scn_array=[cnt:1, {val:1742176616151608003, v:0}], curr_size=122, accum_size=366, time_cost=20705) [2025-03-17 09:56:56.477855] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=13979, avg_log_submit_to_slide_time=13990) [2025-03-17 09:56:56.477903] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:56.478308] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.481014] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=16] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:56.481063] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=44] temporary file module use no blocks [2025-03-17 09:56:56.481073] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=8] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:56.481086] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=11] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:56.481097] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=17] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:56.481101] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=14] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863111372782, flush_io_finished_ret_:0, flush_io_finished_round_:859374, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:56.481124] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=23] temporary file module use no blocks [2025-03-17 09:56:56.481132] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:56.481140] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=7] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:56.481181] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=41] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863111372872, flush_io_finished_ret_:0, flush_io_finished_round_:580424, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:56.481211] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=16] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:56.481242] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=28] temporary file module use no blocks [2025-03-17 09:56:56.481253] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=11] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:56.481260] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=6] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:56.481269] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=9] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863111372970, flush_io_finished_ret_:0, flush_io_finished_round_:580425, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:56.481620] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463483192}], scn_array=[cnt:1, {val:1742176616465845000, v:0}], curr_size=557, accum_size=10780, time_cost=15135) [2025-03-17 09:56:56.482934] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.482964] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=29][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.482982] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=17][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.482990] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7800, ref_cnt_:1, raw_buf_:0x7f7ac6a9a150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0070-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616482553, enqueue_used:2, dequeue_used:17, submit_used:1, return_used:346, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:56.483025] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7800, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0070-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616482553, enqueue_used:2, dequeue_used:17, submit_used:1, return_used:346, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:1742176616483023}}, part_id:-1}) [2025-03-17 09:56:56.483033] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=24][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616482553, enqueue_used:2, dequeue_used:17, submit_used:1, return_used:346, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:122, end_used:-31}}) [2025-03-17 09:56:56.483083] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=47][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:56.483099] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=13][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:56.483372] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.483392] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=19][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.483402] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=9][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a3f2b3510, src_buf_size=295) [2025-03-17 09:56:56.483413] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c67b0, ref_cnt_:1, raw_buf_:0x7f7a3f2b2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0070-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616483112, enqueue_used:1, dequeue_used:14, submit_used:1, return_used:236, callback_enqueue_used:2, callback_dequeue_used:5, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:56.483462] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=40][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c67b0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0070-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616483112, enqueue_used:1, dequeue_used:14, submit_used:1, return_used:236, callback_enqueue_used:2, callback_dequeue_used:5, callback_finish_used:-1, end_used:1742176616483454}}, part_id:-1}) [2025-03-17 09:56:56.483482] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6C-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.483473] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=10][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac862e080, time_log:{begin_ts:1742176616483112, enqueue_used:1, dequeue_used:14, submit_used:1, return_used:236, callback_enqueue_used:2, callback_dequeue_used:5, callback_finish_used:109, end_used:-26}}) [2025-03-17 09:56:56.483502] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6C-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.483502] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=27][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:56.483517] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=13][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac862e080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.483550] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=33][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac862e080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:56.483518] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6C-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c6a0, local_datums:0x7f77cec0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c370, hex: 7D845C1A80300600, int: 1742176616481917}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c378, hex: 7D845C1A80300600, int: 1742176616481917}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.483571] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=18][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:56.483589] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=18][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc5d610}) [2025-03-17 09:56:56.483635] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6C-0-0] [lt=58][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.483618] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=29][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc5e3b8, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:3, rowkey_end_idx:140161962737664, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc5e3b8, micro_handle:NULL}) [2025-03-17 09:56:56.483665] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=46][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:56:56.483674] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=8][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac6b76a58, local_datums:0x7f7ac6b76a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:56:56.483803] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=129][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac6b76a58, local_datums:0x7f7ac6b76a58}, datums:[]}) [2025-03-17 09:56:56.483816] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=13][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176616465845000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:56:56.483825] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=8][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7ac6b75ac0) [2025-03-17 09:56:56.483833] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=7][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:56:56.483886] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA6C-0-0] [lt=29] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2143}, to_seq_no={branch:0, seq:2138}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415383} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.483841] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=7][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1003},{"BIGINT":1};{"BIGINT":1003},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176646381729, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7ac6b753d0, valid:true, source:2, core:{version:{val:1742176616465845000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176616481757, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7ac6b719a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7ac6b756a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7ac6b71848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7ac6b75ac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:56:56.484117] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=276][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:56.484123] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=6][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:56:56.484127] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=4][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:56:56.484159] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=31][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:56:56.484164] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=5][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:56:56.484169] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=4][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:56:56.484173] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=4][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7ac6b6db50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:56:56.484181] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=7][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:56.484206] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0070-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace837300, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519394, assoc_session_id:3222519394, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176616481758, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176616481757, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222519394, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace837300}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.484328] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:146) [3539203][T1004_TntTransf][T1004][YB420A0B093E-00062FB7FC3623F0-0-0] [lt=19] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=3) [2025-03-17 09:56:56.485272] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117878][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA6D-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1742176616485082000, start_id=1742176616485082000, end_id=1742176616485082001) [2025-03-17 09:56:56.486182] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE17-0-0] [lt=18][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:56.486220] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE17-0-0] [lt=37][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.486248] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE17-0-0] [lt=11][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:56.486594] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE17-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6a20ee0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519269, assoc_session_id:3222519269, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176616485082, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176616485100, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ad1c7b118, id:3222519269, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6a20ee0}, exec_ctx.get_errcode()=0) [2025-03-17 09:56:56.487785] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6E-0-0] [lt=1][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.487801] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6E-0-0] [lt=14][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.487811] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6E-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c6a0, local_datums:0x7f77cec0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c370, hex: 0F955C1A80300600, int: 1742176616486159}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c378, hex: 0F955C1A80300600, int: 1742176616486159}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.487925] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA6E-0-0] [lt=55][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.488177] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA6E-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1110}, to_seq_no={branch:0, seq:1105}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415384} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.494996] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA72-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.495015] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA72-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.495023] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA72-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77cec0c690, local_datums:0x7f77cec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77cec0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77cec0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c36e, hex: 61B25C1A80300600, int: 1742176616493665}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c376, hex: 61B25C1A80300600, int: 1742176616493665}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77cec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.495104] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA72-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.495346] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA72-0-0] [lt=34] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415386} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.495408] INFO [SERVER] check_tenant_memory (ob_plugin_vector_index_scheduler.cpp:488) [3539206][T1004_TimerWK0][T1004][YB420A0B093E-00062FB958F5C039-0-0] [lt=53] get vector mem limit size(ret=0, ret="OB_SUCCESS", tenant_id=1004, current_memory_config=0) [2025-03-17 09:56:56.495445] INFO [SERVER] execute_adapter_maintenance (ob_plugin_vector_index_scheduler.cpp:472) [3539206][T1004_TimerWK0][T1004][YB420A0B093E-00062FB958F5C039-0-0] [lt=33] finish generate tenant tablet tasks(ret=0, ret="OB_SUCCESS", tenant_id=1004) [2025-03-17 09:56:56.497750] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5709-0-0] [lt=35][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=10, len=141, pos=123) [2025-03-17 09:56:56.498355] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5709-0-0] [lt=22][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=75, pos=48) [2025-03-17 09:56:56.498940] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add12000 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415387} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1066} write_epoch=1, prev=this=0x7f79add00230 trans_version={val:1742176555817190000, v:0} scn={val:1742176555817190000, v:0} tx_id={txid:14415317} prev=0x7f79add00080 next=(nil) modify_count=3589 acc_checksum=968845299 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4922} write_epoch=3, snapshot_version={val:1742176616497867000, v:0}, *this={this=0x7f7778ff0190 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add00230 latest_compact_node=0x7f78069c74c0 max_trans_version={val:1742176555817190000, v:0} max_trans_id=14415317 max_elr_trans_version={val:1742176555817190000, v:0} max_elr_trans_id=14415317 latest_compact_ts=1742176495202053 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176555817190000, v:0} min_modify_scn={val:1742067188795466000, v:0}}) [2025-03-17 09:56:56.499012] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5709-0-0] [lt=14][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=12, len=142, pos=126) [2025-03-17 09:56:56.498979] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=38][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add12000 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415387} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1066} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646398462, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646398462, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616497867000, v:0}, tx_id:{txid:14415387}, scn:{branch:0, seq:1063}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415387}, tx_desc:{this:0x7f7ae6be8340, tx_id:{txid:14415387}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616497867, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616496851, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1066}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.499036] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415387}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1066}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.499050] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant clog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415387} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.499071] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.499080] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: FAC25C1A80300600, int: 1742176616497914}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: FAC25C1A80300600, int: 1742176616497914}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 5689617100000000, int: 1902217558}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 5689617100000000, int: 1902217558}]}) [2025-03-17 09:56:56.499163] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=49][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.499520] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA74-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415387} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.503524] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA76-0-0] [lt=8][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.503544] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA76-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.503554] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA76-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: B3D35C1A80300600, int: 1742176616502195}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: B3D35C1A80300600, int: 1742176616502195}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.503633] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA76-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.503808] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA76-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1072}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415388} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.507027] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add126b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415389} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1074} write_epoch=1, prev=this=0x7f79add008e8 trans_version={val:1742176555850152000, v:0} scn={val:1742176555850152000, v:0} tx_id={txid:14415319} prev=0x7f79add00738 next=(nil) modify_count=3589 acc_checksum=1616903435 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2920} write_epoch=3, snapshot_version={val:1742176616506390000, v:0}, *this={this=0x7f7778ff0888 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add008e8 latest_compact_node=0x7f78069c7588 max_trans_version={val:1742176555850152000, v:0} max_trans_id=14415319 max_elr_trans_version={val:1742176555850152000, v:0} max_elr_trans_id=14415319 latest_compact_ts=1742176495269304 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176555850152000, v:0} min_modify_scn={val:1742067188820920000, v:0}}) [2025-03-17 09:56:56.507060] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=32][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add126b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415389} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1074} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646406655, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646406655, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616506390000, v:0}, tx_id:{txid:14415389}, scn:{branch:0, seq:1071}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415389}, tx_desc:{this:0x7f7ae6a88200, tx_id:{txid:14415389}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616506390, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616505394, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1074}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.507116] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415389}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1074}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.507126] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=10][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant clog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415389} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.507170] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=43][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.507178] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: 61E45C1A80300600, int: 1742176616506465}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: 61E45C1A80300600, int: 1742176616506465}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 57EE622900000000, int: 694349399}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 57EE622900000000, int: 694349399}]}) [2025-03-17 09:56:56.507234] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.507527] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA78-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2057}, to_seq_no={branch:0, seq:1072}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415389} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.508175] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=46] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=25124, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:56:56.508207] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8875) [2025-03-17 09:56:56.510667] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=40] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:56.510704] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=8] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:56.511172] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7A-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.511190] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7A-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.511198] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36f, hex: 20F15C1A80300600, int: 1742176616509728}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c377, hex: 20F15C1A80300600, int: 1742176616509728}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37f, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c387, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.511285] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7A-0-0] [lt=49][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.511495] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7A-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1069}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415390} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.515037] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7C-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.515063] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7C-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.515073] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7C-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: BB015D1A80300600, int: 1742176616513979}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: BB015D1A80300600, int: 1742176616513979}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0060000000000000, int: 24576}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0060000000000000, int: 24576}]}) [2025-03-17 09:56:56.515191] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA7C-0-0] [lt=53][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.515390] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117878][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7C-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1864}, to_seq_no={branch:0, seq:800}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415391} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.518980] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117902][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7E-0-0] [lt=28][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.519000] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117902][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7E-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.519008] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117902][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7E-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f7ad8e0c690, local_datums:0x7f7ad8e0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f7ad8e0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f7ad8e0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c36f, hex: 77125D1A80300600, int: 1742176616518263}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c377, hex: 77125D1A80300600, int: 1742176616518263}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c37f, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f7ad8e0c387, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.519075] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117902][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7E-0-0] [lt=36][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.519294] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA7E-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1943}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415392} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.522676] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add131b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415393} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:1958} write_epoch=1, prev=this=0x7f77ee733d48 trans_version={val:1742161917788906001, v:0} scn={val:1742161917788906001, v:0} tx_id={txid:14148823} prev=0x7f789e360b60 next=(nil) modify_count=17 acc_checksum=1934688301 version=1742067188266264 type=1 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:5268} write_epoch=0, snapshot_version={val:1742176616521281000, v:0}, *this={this=0x7f7778ff14b0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x7f77ee733d48 latest_compact_node=0x7f77ee733d48 max_trans_version={val:1742176555942543000, v:0} max_trans_id=14415323 max_elr_trans_version={val:1742176555942543000, v:0} max_elr_trans_id=14415323 latest_compact_ts=1742161917791497 last_compact_cnt=17 total_trans_node_cnt=18 max_modify_scn={val:1742161917788906001, v:0} min_modify_scn={val:1742109728996641000, v:0}}) [2025-03-17 09:56:56.522724] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=48][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add131b0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415393} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:1958} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646422229, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646422229, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616521281000, v:0}, tx_id:{txid:14415393}, scn:{branch:0, seq:1057}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415393}, tx_desc:{this:0x7f7ae6a14ca0, tx_id:{txid:14415393}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616522343, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616520386, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1958}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.522781] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415393}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1958}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.522791] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=10][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant local data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415393} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.522811] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.522817] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=4][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36f, hex: E31E5D1A80300600, int: 1742176616521443}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c377, hex: E31E5D1A80300600, int: 1742176616521443}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37f, hex: 00E0BA0100000000, int: 29024256}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c387, hex: 00E0BA0100000000, int: 29024256}]}) [2025-03-17 09:56:56.522883] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=50][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.523122] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA80-0-0] [lt=23] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2120}, to_seq_no={branch:0, seq:1058}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415393} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.526618] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA82-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.526638] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA82-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.526646] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA82-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36f, hex: 562F5D1A80300600, int: 1742176616525654}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c377, hex: 562F5D1A80300600, int: 1742176616525654}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37f, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c387, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.526741] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA82-0-0] [lt=49][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.526966] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117878][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA82-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2094}, to_seq_no={branch:0, seq:1036}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415394} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.562305] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:56.562354] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=25] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:56.571648] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463490458}], scn_array=[cnt:1, {val:1742176616527601000, v:0}], curr_size=555, accum_size=18044, time_cost=43342) [2025-03-17 09:56:56.573079] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162411320}], scn_array=[cnt:1, {val:1742176616151608004, v:0}], curr_size=122, accum_size=488, time_cost=15943) [2025-03-17 09:56:56.573175] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=16][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859672097}], scn_array=[cnt:1, {val:1742176616529737000, v:0}], curr_size=588, accum_size=1886, time_cost=42549) [2025-03-17 09:56:56.573364] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872699175}], scn_array=[cnt:1, {val:1742176616151608004, v:0}], curr_size=122, accum_size=488, time_cost=16182) [2025-03-17 09:56:56.573375] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=36] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1742176616151608002, v:0}, new_max_version={val:1742176616151608002, v:0}, last_min_version={val:1742176616151608000, v:0}, last_max_version={val:1742176616151608000, v:0}, total_time_us=44245, delta_us=421766) [2025-03-17 09:56:56.573401] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=23] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1742176616151608002, v:0}, min_version={val:1742176616151608002, v:0}, max_version={val:1742176616151608002, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:56:56.573430] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:56:56.574014] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA84-0-0] [lt=12][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.574038] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA84-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.574048] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA84-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36f, hex: 8EE65D1A80300600, int: 1742176616572558}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c377, hex: 8EE65D1A80300600, int: 1742176616572558}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37f, hex: 0060000000000000, int: 24576}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c387, hex: 0060000000000000, int: 24576}]}) [2025-03-17 09:56:56.574193] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA84-0-0] [lt=56][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.574495] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA84-0-0] [lt=49] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2782}, to_seq_no={branch:0, seq:1725}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415395} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.578381] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.583925] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463491013}], scn_array=[cnt:1, {val:1742176616543520000, v:0}], curr_size=587, accum_size=18631, time_cost=12189) [2025-03-17 09:56:56.584051] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40303021260, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:56.594519] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463491600}], scn_array=[cnt:1, {val:1742176616576393000, v:0}], curr_size=555, accum_size=19186, time_cost=10518) [2025-03-17 09:56:56.595822] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=7] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:56.597292] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=21][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.597320] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=27][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.597335] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=15][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.597341] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5fd0, ref_cnt_:1, raw_buf_:0x7f7aa4262150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F2-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176616597017, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:236, callback_enqueue_used:3, callback_dequeue_used:17, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.597378] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=28][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5fd0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F2-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176616597017, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:236, callback_enqueue_used:3, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176616597371}}, part_id:-1}) [2025-03-17 09:56:56.597390] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=40][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:56.597621] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.597643] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=21][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.597649] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d00f15a, src_buf_size=1611) [2025-03-17 09:56:56.597656] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6918, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F2-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176616597448, enqueue_used:1, dequeue_used:10, submit_used:0, return_used:155, callback_enqueue_used:1, callback_dequeue_used:5, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.597688] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=25][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6918, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F2-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176616597448, enqueue_used:1, dequeue_used:10, submit_used:0, return_used:155, callback_enqueue_used:1, callback_dequeue_used:5, callback_finish_used:-1, end_used:1742176616597682}}, part_id:-1}) [2025-03-17 09:56:56.597761] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222510318, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf35c960}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:56.597791] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=21] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634278} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:56.597823] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=28] end stmt(ret=0, tx_id=9634278, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf35c960, tx_id:{txid:9634278}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222510318, assoc_session_id:3222510318, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176616596548, active_ts:1742176616596548, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646596542, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460711213107, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616596551}], exec_info_reap_ts:1742176616596550, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616596547, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222510318, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf35c960}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.597935] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634278}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176646596542, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616596548}{self_ls_id:{id:1}, session_id:3222510318, associated_session_id:3222510318, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634278}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460711213107, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616596548, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634278} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.597996] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=53] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634278}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176646596542, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616596548}{self_ls_id:{id:1}, session_id:3222510318, associated_session_id:3222510318, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634278}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460711213107, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616596548, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634278} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.598033] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=34] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634278}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176616597932, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:56.598047] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=13] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf35c960, tx_id:{txid:9634278}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222510318, assoc_session_id:3222510318, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176616596548, active_ts:1742176616596548, commit_ts:-1, finish_ts:1742176616597604, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646596542, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460711213107, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616596551}], exec_info_reap_ts:1742176616596550, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616596547, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:56.598110] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F2-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:56.601804] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA88-0-0] [lt=16][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.601825] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA88-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.601838] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA88-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: 1F525E1A80300600, int: 1742176616600095}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: 1F525E1A80300600, int: 1742176616600095}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.601963] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA88-0-0] [lt=68][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.602257] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA88-0-0] [lt=34] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1727}, to_seq_no={branch:0, seq:1064}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415397} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.606909] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8A-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.606934] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8A-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.606942] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: A1665E1A80300600, int: 1742176616605345}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: A1665E1A80300600, int: 1742176616605345}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.607033] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8A-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.607317] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA8A-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2869}, to_seq_no={branch:0, seq:2117}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415398} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.611753] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add13fd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415399} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1863} write_epoch=1, prev=this=0x7f79add02038 trans_version={val:1742176556031241000, v:0} scn={val:1742176556031241000, v:0} tx_id={txid:14415329} prev=0x7f79add01e88 next=(nil) modify_count=3589 acc_checksum=1979706618 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3225} write_epoch=3, snapshot_version={val:1742176616610346000, v:0}, *this={this=0x7f7778ff2190 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add02038 latest_compact_node=0x7f78069c7650 max_trans_version={val:1742176556031241000, v:0} max_trans_id=14415329 max_elr_trans_version={val:1742176556031241000, v:0} max_elr_trans_id=14415329 latest_compact_ts=1742176495494929 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176556031241000, v:0} min_modify_scn={val:1742067189120347000, v:0}}) [2025-03-17 09:56:56.611807] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=53][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add13fd8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415399} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1863} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646511287, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646511287, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616610346000, v:0}, tx_id:{txid:14415399}, scn:{branch:0, seq:1058}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415399}, tx_desc:{this:0x7f7ae6ba3680, tx_id:{txid:14415399}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616611411, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616609549, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1863}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.611882] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=74][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415399}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1863}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.611901] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=18][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant slog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415399} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.611933] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=31][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.611945] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: 2F7B5E1A80300600, int: 1742176616610607}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: 2F7B5E1A80300600, int: 1742176616610607}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0030CD0200000000, int: 47001600}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0030CD0200000000, int: 47001600}]}) [2025-03-17 09:56:56.612025] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=57][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.612280] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA8C-0-0] [lt=43] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2124}, to_seq_no={branch:0, seq:1059}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415399} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.616617] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8E-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.616635] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8E-0-0] [lt=17][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.616642] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8E-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: D68F5E1A80300600, int: 1742176616615894}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: D68F5E1A80300600, int: 1742176616615894}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.616713] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA8E-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.616893] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA8E-0-0] [lt=69] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1881}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415400} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.620401] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=9][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f778f2cdcf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415401} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1870} write_epoch=1, prev=this=0x7f79add026f0 trans_version={val:1742176556040875000, v:0} scn={val:1742176556040875000, v:0} tx_id={txid:14415331} prev=0x7f79add02540 next=(nil) modify_count=3589 acc_checksum=2166463503 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3689} write_epoch=3, snapshot_version={val:1742176616619881000, v:0}, *this={this=0x7f7778ff2888 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add026f0 latest_compact_node=0x7f78069c7718 max_trans_version={val:1742176556040875000, v:0} max_trans_id=14415331 max_elr_trans_version={val:1742176556040875000, v:0} max_elr_trans_id=14415331 latest_compact_ts=1742176495504834 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176556040875000, v:0} min_modify_scn={val:1742067189136318000, v:0}}) [2025-03-17 09:56:56.620462] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=61][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f778f2cdcf0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415401} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1870} write_epoch=1, ctx={this:0x7f7af49cf658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646519985, tablet_id:{id:309}, table_iter:0x7f7af49cdb48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646519985, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616619881000, v:0}, tx_id:{txid:14415401}, scn:{branch:0, seq:1867}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415401}, tx_desc:{this:0x7f7acaf8f2c0, tx_id:{txid:14415401}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616619881, commit_ts:-1, finish_ts:-1, timeout_us:29999996, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616618015, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b95d950, mem_ctx:0x7f7b0b95eed0, tx_scn:{branch:0, seq:1870}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.620536] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=74][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415401}, data:0x7f7af49cc508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1870}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.620552] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant slog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415401} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.620583] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=30][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.620592] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77c0e0c690, local_datums:0x7f77c0e0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77c0e0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77c0e0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c36e, hex: 429C5E1A80300600, int: 1742176616619074}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c376, hex: 429C5E1A80300600, int: 1742176616619074}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c37e, hex: 0010E30300000000, int: 65212416}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c386, hex: 0010E30300000000, int: 65212416}]}) [2025-03-17 09:56:56.620656] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117908][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=46][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.620919] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA90-0-0] [lt=52] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2173}, to_seq_no={branch:0, seq:1868}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415401} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.627991] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=24956928) [2025-03-17 09:56:56.628038] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=5] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:56.628707] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA94-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.628734] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA94-0-0] [lt=26][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.628744] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA94-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f779fc0c690, local_datums:0x7f779fc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f779fc0c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f779fc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c36e, hex: B4BD5E1A80300600, int: 1742176616627636}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c376, hex: B4BD5E1A80300600, int: 1742176616627636}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f779fc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.628838] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA94-0-0] [lt=56][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.629080] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA94-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3206}, to_seq_no={branch:0, seq:2138}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415403} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.633896] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add14d48 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415404} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1745} write_epoch=1, prev=this=0x7f77ee737980 trans_version={val:1742161918111448000, v:0} scn={val:1742161918111448000, v:0} tx_id={txid:14148834} prev=0x7f77ee737640 next=(nil) modify_count=17 acc_checksum=2173273780 version=1742067188266264 type=1 flag=4 snapshot_barrier=1742176615942847000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4252} write_epoch=0, snapshot_version={val:1742176616632600000, v:0}, *this={this=0x7f7778ff3148 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x7f77ee737980 latest_compact_node=0x7f77ee737980 max_trans_version={val:1742176556067421000, v:0} max_trans_id=14415334 max_elr_trans_version={val:1742176556067421000, v:0} max_elr_trans_id=14415334 latest_compact_ts=1742161918146098 last_compact_cnt=17 total_trans_node_cnt=18 max_modify_scn={val:1742161918111448000, v:0} min_modify_scn={val:1742109729202156000, v:0}}) [2025-03-17 09:56:56.633943] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=46][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add14d48 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14415404} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1745} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176646533399, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176646533399, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176616632600000, v:0}, tx_id:{txid:14415404}, scn:{branch:0, seq:1064}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14415404}, tx_desc:{this:0x7f7ae6b93380, tx_id:{txid:14415404}, state:1, addr:"192.168.1.65:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176616633657, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616631913, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1745}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:56.634025] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=81][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14415404}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1745}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:56:56.634039] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.65", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":3},{"VARCHAR":"tenant meta data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14415404} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:56:56.634065] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.634074] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77ec00c690, local_datums:0x7f77ec00c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77ec00c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77ec00c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c36e, hex: 91D25E1A80300600, int: 1742176616632977}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c376, hex: 91D25E1A80300600, int: 1742176616632977}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c37e, hex: 0090B90000000000, int: 12161024}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c386, hex: 0090B90000000000, int: 12161024}]}) [2025-03-17 09:56:56.634138] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=41][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.634444] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA96-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2130}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415404} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.635709] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.635714] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636014] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=11] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636282] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636471] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636482] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636490] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636494] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=22] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636722] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.636781] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:56.641307] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=31] gc stale ls task succ [2025-03-17 09:56:56.646775] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=4) [2025-03-17 09:56:56.648629] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:257) [3539431][T1003_ArcSender][T1003][YB420A0B093E-00062FB7F8D54125-0-0] [lt=25][errcode=-4018] try free send task failed(ret=-4018) [2025-03-17 09:56:56.657208] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=2455, avg_group_log_size=223) [2025-03-17 09:56:56.657241] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=2455) [2025-03-17 09:56:56.657270] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=9, avg_log_batch_cnt=1, total_group_log_size=1098, avg_group_log_size=122) [2025-03-17 09:56:56.657283] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1098) [2025-03-17 09:56:56.660034] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463497219}], scn_array=[cnt:1, {val:1742176616634719000, v:0}], curr_size=552, accum_size=24802, time_cost=24330) [2025-03-17 09:56:56.661953] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA98-0-0] [lt=36][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.661986] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA98-0-0] [lt=31][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.662000] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA98-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77ec00c690, local_datums:0x7f77ec00c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77ec00c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77ec00c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c36e, hex: 783E5F1A80300600, int: 1742176616660600}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c376, hex: 783E5F1A80300600, int: 1742176616660600}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.662136] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA98-0-0] [lt=62][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.662525] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117897][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA98-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2804}, to_seq_no={branch:0, seq:1739}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415405} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.669425] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:817) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=25] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}) [2025-03-17 09:56:56.673830] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=39] gc stale ls task succ [2025-03-17 09:56:56.678496] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.678687] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [117536][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=25] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=518) [2025-03-17 09:56:56.681718] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463497771}], scn_array=[cnt:1, {val:1742176616663406000, v:0}], curr_size=553, accum_size=25355, time_cost=17682) [2025-03-17 09:56:56.683693] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9A-0-0] [lt=29][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.683717] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9A-0-0] [lt=22][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.683730] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9A-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77ec00c690, local_datums:0x7f77ec00c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77ec00c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f77ec00c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c36e, hex: 7B915F1A80300600, int: 1742176616681851}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c376, hex: 7B915F1A80300600, int: 1742176616681851}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c37e, hex: 0060400000000000, int: 4218880}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c386, hex: 0060400000000000, int: 4218880}]}) [2025-03-17 09:56:56.683879] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9A-0-0] [lt=71][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.684162] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA9A-0-0] [lt=34] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2129}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415406} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.696765] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=39] current monitor number(seq_=-1) [2025-03-17 09:56:56.706697] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:56.707647] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=40] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.193809866905212402e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=596, Throttle Trigger(MB)=358, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.394460944325214200e+02) [2025-03-17 09:56:56.728615] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463498324}], scn_array=[cnt:1, {val:1742176616684641000, v:0}], curr_size=553, accum_size=25908, time_cost=42966) [2025-03-17 09:56:56.728731] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=24][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.728767] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=36][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.728789] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:56.728799] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5ac0, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F3-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176616707959, enqueue_used:1, dequeue_used:40, submit_used:1, return_used:20714, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.728867] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=58][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5ac0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F3-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176616707959, enqueue_used:1, dequeue_used:40, submit_used:1, return_used:20714, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:1742176616728859}}, part_id:-1}) [2025-03-17 09:56:56.729159] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=33][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:56.729196] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=36][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:56.729207] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=10][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d00f15a, src_buf_size=1611) [2025-03-17 09:56:56.729219] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c82f8, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F3-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176616728898, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:208, callback_enqueue_used:1, callback_dequeue_used:27, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:56.729268] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=42][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c82f8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F3-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176616728898, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:208, callback_enqueue_used:1, callback_dequeue_used:27, callback_finish_used:-1, end_used:1742176616729263}}, part_id:-1}) [2025-03-17 09:56:56.729378] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222519572, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3ab800}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:56.729414] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:21566}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634279} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:56.729458] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=42] end stmt(ret=0, tx_id=9634279, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3ab800, tx_id:{txid:9634279}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519572, assoc_session_id:3222519572, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176616706924, active_ts:1742176616706924, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646706918, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460822096311, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616706927}], exec_info_reap_ts:1742176616706926, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616706923, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222519572, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3ab800}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:56.729568] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634279}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176646706918, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616706924}{self_ls_id:{id:1}, session_id:3222519572, associated_session_id:3222519572, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634279}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460822096311, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616706924, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634279} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.729638] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=59] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634279}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176646706918, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176616706924}{self_ls_id:{id:1}, session_id:3222519572, associated_session_id:3222519572, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634279}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409460822096311, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176616706924, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634279} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:56.729681] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=39] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634279}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176616729564, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:56.729694] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=12] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3ab800, tx_id:{txid:9634279}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519572, assoc_session_id:3222519572, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176616706924, active_ts:1742176616706924, commit_ts:-1, finish_ts:1742176616729359, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176646706918, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409460822096311, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176616706927}], exec_info_reap_ts:1742176616706926, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176616706923, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:56.729794] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F3-0-0] [lt=1][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:56.730756] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9C-0-0] [lt=22][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:56:56.730775] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9C-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:56:56.730784] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9C-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77ec00c690, local_datums:0x7f77ec00c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77ec00c344, hex: 31302E31312E392E3635, cstr: 192.168.1.65}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c356, hex: 0300000000000000, int: 3}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f77ec00c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c36d, hex: A64B601A80300600, int: 1742176616729510}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c375, hex: A64B601A80300600, int: 1742176616729510}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77ec00c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:56:56.730855] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B0941-00063020D07CFA9C-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:56:56.731115] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B0941-00063020D07CFA9C-0-0] [lt=29] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2087}, to_seq_no={branch:0, seq:1025}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14415407} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:56:56.744339] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=27044) [2025-03-17 09:56:56.747481] INFO [CLOG] update_upstream (ob_remote_location_adaptor.cpp:91) [3539230][T1004_LogRessvr][T1004][YB420A0B093E-00062FB7F7C54125-0-0] [lt=26] primary tenant, just skip [2025-03-17 09:56:56.772567] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463498877}], scn_array=[cnt:1, {val:1742176616732588000, v:0}], curr_size=549, accum_size=26457, time_cost=39159) [2025-03-17 09:56:56.778026] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162411564}], scn_array=[cnt:1, {val:1742176616652787001, v:0}], curr_size=122, accum_size=732, time_cost=20746) [2025-03-17 09:56:56.778103] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=20435, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:56:56.778129] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872699419}], scn_array=[cnt:1, {val:1742176616652787001, v:0}], curr_size=122, accum_size=732, time_cost=20845) [2025-03-17 09:56:56.778169] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=26, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:56:56.778191] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=6, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:56:56.778210] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=24] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:56.778217] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=20563, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:56:56.778237] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=28, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:56:56.778540] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.783907] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=84] [PALF STAT WRITE LOG COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=13314, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:56:56.783933] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463499426}], scn_array=[cnt:1, {val:1742176616744264000, v:0}], curr_size=587, accum_size=27044, time_cost=11186) [2025-03-17 09:56:56.783958] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=27044) [2025-03-17 09:56:56.783972] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=14, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:56:56.784003] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=27044) [2025-03-17 09:56:56.784030] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=7, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:56:56.784035] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT LOG LIFE TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=16189, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:56:56.784046] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=5] [PALF STAT FLUSH CB COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=32, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:56:56.784364] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40303021260, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:56.794275] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=6358, avg_group_log_size=454) [2025-03-17 09:56:56.794335] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Slow write(duration=10606, write_len=4096) [2025-03-17 09:56:56.794362] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107777, offset=57285294}, end_cursor:ObLogCursor{file_id=5, log_id=1107778, offset=57285600}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b176f8050, len:1362, log_data_len:306, seq:1107777, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:56.794511] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:56:56.794548] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176616652787000, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176616652787000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:56.796228] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=13][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:56.796526] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=25] get number(ret=0, service_type_=0, range=1, base_id=1742176616796101000, start_id=1742176616796101000, end_id=1742176616796101001) [2025-03-17 09:56:56.835832] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859673861}], scn_array=[cnt:1, {val:1742176616793980000, v:0}], curr_size=588, accum_size=3650, time_cost=41548) [2025-03-17 09:56:56.835972] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=13] [APPLY STAT CB APPEND COST TIME](cur_stat_count=58, stat_interval=5000000, avg (count/cost)=39, this=0x7f798c66c280, extra_info="") [2025-03-17 09:56:56.836000] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=27] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=58, stat_interval=5000000, avg (count/cost)=26946, this=0x7f798c66c4b0, extra_info="") [2025-03-17 09:56:56.836008] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=58, stat_interval=5000000, avg (count/cost)=0, this=0x7f798c66c6e0, extra_info="") [2025-03-17 09:56:56.836015] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB EXECUTE TIME](cur_stat_count=58, stat_interval=5000000, avg (count/cost)=-476, this=0x7f798c66c910, extra_info="") [2025-03-17 09:56:56.836021] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB TOTAL TIME](cur_stat_count=58, stat_interval=5000000, avg (count/cost)=26509, this=0x7f798c66cb40, extra_info="") [2025-03-17 09:56:56.846885] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=9] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=3) [2025-03-17 09:56:56.855184] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859674449}], scn_array=[cnt:1, {val:1742176616796101000, v:0}], curr_size=1132, accum_size=4782, time_cost=19263) [2025-03-17 09:56:56.855318] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.857309] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae329470, extra_info="") [2025-03-17 09:56:56.857358] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=33] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:56:56.857492] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=22] QSched: |cfg: root:1,x,0 net_in:1,1249M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:428K/98:16290 disk:428K/98:16290 t1m2:240K/53:13950 t1m2g0:240K/53:13950 t1004m2:88K/22:18121 t1004m2g0:88K/22:18121 t1003m2:100K/23:19933 t1003m2g0:100K/23:19933 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:56:56.857789] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.862844] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107778, offset=57285600}, end_cursor:ObLogCursor{file_id=5, log_id=1107779, offset=57285897}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17714050, len:1056, log_data_len:297, seq:1107778, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:56.863009] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=21] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:56:56.863053] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7318-0-0] [lt=30] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176616652787001, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176616652787001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:56.878127] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859675581}], scn_array=[cnt:1, {val:1742176616864070000, v:0}], curr_size=749, accum_size=5531, time_cost=13203) [2025-03-17 09:56:56.878612] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.895230] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.920297] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4006] Rpc proxy not inited(ret=-4006) [2025-03-17 09:56:56.923065] WDIAG [SHARE] get_restore_shared_storage_limit (ob_backup_connectivity.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=31][errcode=-4018] no exist row(ret=-4018, sql=SELECT max_iops, max_bandwidth FROM __all_backup_storage_info WHERE tenant_id = 1004 AND dest_id = '1') [2025-03-17 09:56:56.926485] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=23][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926538] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=52][errcode=0] update shared device limit failed(ret=0, i=2, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926571] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=32][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926598] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=26][errcode=0] update shared device limit failed(ret=0, i=5, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.926621] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=21][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926650] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=28][errcode=0] update shared device limit failed(ret=0, i=8, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926670] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=18][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926694] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=23][errcode=0] update shared device limit failed(ret=0, i=11, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.926718] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=21][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926733] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=14][errcode=0] update shared device limit failed(ret=0, i=14, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926748] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=14][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926763] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=15][errcode=0] update shared device limit failed(ret=0, i=17, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.926780] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=16][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926795] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=15][errcode=0] update shared device limit failed(ret=0, i=20, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926816] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=19][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926836] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=20][errcode=0] update shared device limit failed(ret=0, i=23, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.926856] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=17][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926876] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=20][errcode=0] update shared device limit failed(ret=0, i=26, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926902] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=24][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926919] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=17][errcode=0] update shared device limit failed(ret=0, i=29, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.926942] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=21][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.926962] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=20][errcode=0] update shared device limit failed(ret=0, i=32, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.926984] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=20][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.927005] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=20][errcode=0] update shared device limit failed(ret=0, i=35, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.927032] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=25][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.927050] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=17][errcode=0] update shared device limit failed(ret=0, i=38, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:56.927093] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=41][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:56.927115] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE48-0-0] [lt=21][errcode=0] update shared device limit failed(ret=0, i=41, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:56.927259] INFO pn_ratelimit (group.c:653) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=33] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2025-03-17 09:56:56.927279] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117408][TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=28] trace max work area(auto_calc=true, tenant_max_memory_limit=1717986918, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=85899345, max_wa_memory_size=85899345, tmp_max_wa_memory_size=85899345, pre_mem_target=85899345, remain_memory_size=85899345, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=1443631104, washable_size=-2, max_workarea_memory_size=85899345, max_tenant_memory_size=274355814, wash_ratio=6, tenant_id=1003) [2025-03-17 09:56:56.927324] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:56:56.927342] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:56:56.927339] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117408][TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=33] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=10737418, calc_global_bound_size=1048576000000, mem_target_=85899345, wa_max_memory_size=85899345, profile_cnt_=0, total_memory_size=0, max_wa_size=85899345, avg_bound_size=10737418, max_bound_size=10737418, min_bound_size_=1048576) [2025-03-17 09:56:56.927353] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117408][TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, global_bound_size_=10737418, manual_calc_cnt_=44644, drift_size_=0, pre_drift_size=0, wa_max_memory_size=85899345, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=10737418, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=85899345, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2025-03-17 09:56:56.927408] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117408][TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=27] trace max work area(auto_calc=true, tenant_max_memory_limit=15461882266, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=773094113, max_wa_memory_size=773094113, tmp_max_wa_memory_size=773094113, pre_mem_target=773094113, remain_memory_size=773094113, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=7391244288, washable_size=-2, max_workarea_memory_size=773094113, max_tenant_memory_size=8070637978, wash_ratio=6, tenant_id=1004) [2025-03-17 09:56:56.927466] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117408][TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=33] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=96636764, calc_global_bound_size=1048576000000, mem_target_=773094113, wa_max_memory_size=773094113, profile_cnt_=0, total_memory_size=0, max_wa_size=773094113, avg_bound_size=96636764, max_bound_size=96636764, min_bound_size_=1048576) [2025-03-17 09:56:56.927477] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117408][TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=10] timer to calc global bound size(ret=0, global_bound_size_=96636764, manual_calc_cnt_=492, drift_size_=0, pre_drift_size=0, wa_max_memory_size=773094113, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=96636764, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=773094113, auto_calc=true, sql_mem_callback_.get_total_dump_size()=69192448) [2025-03-17 09:56:56.937196] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=8] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:0, threads_idle_time:1312797256620, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:56.937340] INFO [LIB] do_thread_recycle (threads.cpp:163) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=43] recycle one thread(this=0x7f7b319af170, total=4, remain=3) [2025-03-17 09:56:56.937355] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=14] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:0, threads_idle_time:1312797256620, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:56.941649] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=19] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:56:56.941904] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175478][T1_L0_G5][T1][YB420A0B0940-00063024191DDEFD-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1742176616941722000, start_id=1742176616941722000, end_id=1742176616941722001) [2025-03-17 09:56:56.945138] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:56:56.945213] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=57] [PALF STAT APPEND COST TIME](cur_stat_count=46, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:56:56.957394] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:56:56.958879] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:202) [3539227][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=4] query dest_info_ succ(dest_info=[[0, "file:///home/admin/backup/sgkydb/1741272511/tenant_incarnation_1/1004/clog"]], archive_dest_changed=false) [2025-03-17 09:56:56.958905] INFO [CLOG.EXTLOG] resize_log_ext_handler_ (ob_cdc_service.cpp:750) [3539227][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=24] finish to resize log external storage handler(current_ts=1742176616958903, tenant_max_cpu=8, valid_ls_v1_count=0, valid_ls_v2_count=0, other_ls_count=0, new_concurrency=0) [2025-03-17 09:56:56.958921] INFO [CLOG.EXTLOG] run1 (ob_cdc_service.cpp:241) [3539227][T1004_CdcSrv][T1004][Y0-0000000000000000-0-0] [lt=14] snapshot_traffic_info once [2025-03-17 09:56:56.961409] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463500135}], scn_array=[cnt:1, {val:1742176616944918000, v:0}], curr_size=587, accum_size=709, time_cost=16190) [2025-03-17 09:56:56.962614] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.966692] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.968540] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [117761][T1_Occam][T1][YB420A0B093E-00062FB860254125-0-0] [lt=33] ====== tenant freeze timer task ====== [2025-03-17 09:56:56.969260] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C84-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:56.974671] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA582-0-0] [lt=8][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:56.978673] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:56.984708] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=8] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40294612172, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:56.993588] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=48] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:56:57.004001] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB906EB5C8D-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.010795] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:57.010857] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=15] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:57.011758] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3182-0-0] [lt=28] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12274, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:57.011983] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3182-0-0] [lt=20] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12274, locations=[{table_id:12274, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.012013] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3182-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:12274, add_timestamp:1742176617011718}, locations=[{table_id:12274, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}, {table_id:12274, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617011968, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.027856] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=52][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859676918}], scn_array=[cnt:1, {val:1742176616992788000, v:0}], curr_size=588, accum_size=6707, time_cost=34258) [2025-03-17 09:56:57.046990] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=50] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=2) [2025-03-17 09:56:57.057481] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:56:57.062452] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=74] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:57.062526] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=24] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:57.073697] INFO [CLOG] is_in_sync (ob_log_handler.cpp:709) [117882][T1_L5_G0][T1][YB420A0B0940-00063020D00CE197-0-0] [lt=25] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2025-03-17 09:56:57.076101] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872699785}], scn_array=[cnt:1, {val:1742176616652787004, v:0}], curr_size=122, accum_size=1098, time_cost=18629) [2025-03-17 09:56:57.078734] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.107867] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=8117) [2025-03-17 09:56:57.118195] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:56:57.118217] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=30] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:56:57.118274] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=38] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:20, fail_bw:81788, [delay/us]:prepare:1, schedule:19, submit:1, rt:1470, total:1491, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:57.118317] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=26] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4642, iops:45, bw:208561, [delay/us]:prepare:5, schedule:20, submit:71, rt:12690, total:12787) [2025-03-17 09:56:57.118342] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=12, iops=12, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:57.118502] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=152] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:20, avg_bw:81775, [delay/us]: prepare:1, schedule:19, submit:1, device:1470, total:1491) [2025-03-17 09:56:57.118536] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4642, avg_iops:45, avg_bw:208523, [delay/us]: prepare:5, schedule:20, submit:71, device:12690, total:12787) [2025-03-17 09:56:57.118576] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:56:57.118640] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32707, [delay/us]:prepare:1, schedule:19, submit:1, rt:9474, total:9496, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:57.118672] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4608, iops:16, bw:73592, [delay/us]:prepare:7, schedule:30, submit:16, rt:22234, total:22289) [2025-03-17 09:56:57.118691] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS TENANT](tenant_id=1003, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:57.118754] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=57] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32705, [delay/us]: prepare:1, schedule:19, submit:1, device:9474, total:9496) [2025-03-17 09:56:57.118785] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4608, avg_iops:16, avg_bw:73587, [delay/us]: prepare:7, schedule:30, submit:16, device:22234, total:22289) [2025-03-17 09:56:57.118808] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:56:57.118853] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8176, [delay/us]:prepare:5, schedule:20, submit:1, rt:8948, total:8975) [2025-03-17 09:56:57.118894] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:20, bw:81760, [delay/us]:prepare:6, schedule:23, submit:1952, rt:10126, total:12108) [2025-03-17 09:56:57.118914] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:57.118977] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=58] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8175, [delay/us]: prepare:5, schedule:20, submit:1, device:8948, total:8975) [2025-03-17 09:56:57.119006] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4096, avg_iops:20, avg_bw:81754, [delay/us]: prepare:6, schedule:23, submit:1952, device:10126, total:12108) [2025-03-17 09:56:57.119031] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:56:57.119086] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=33] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4087, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:358, total:364) [2025-03-17 09:56:57.119127] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:57.119204] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=72] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4087, [delay/us]: prepare:0, schedule:0, submit:600000000, device:358, total:364) [2025-03-17 09:56:57.119236] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:56:57.119265] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:57.119284] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119302] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119317] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119330] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119342] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119360] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119373] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119391] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:57.119432] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=39] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119439] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119451] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119466] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:57.119485] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS SERVER] net_in=1038kB/s, net_out=246kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:56:57.120085] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7F0365901-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.120162] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117789][T1_DupTbLease][T1][YB420A0B093E-00062FB7F0365901-0-0] [lt=26][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.120568] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117789][T1_DupTbLease][T1][YB420A0B093E-00062FB7F0365901-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ad3352780, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222500436, assoc_session_id:3222500436, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617119286, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617117167, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7888660038, id:3222500436, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ad3352780}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.120670] WDIAG [STORAGE.DUP_TABLE] execute_for_dup_ls_ (ob_dup_table_util.cpp:207) [117789][T1_DupTbLease][T1][Y0-0000000000000000-0-0] [lt=0][errcode=0] refresh dup ls failed(tmp_ret=-4016, this={tenant_id_:1, dup_table_scan_timer_:0x7f7b0ac3dae8, dup_loop_worker_:0x7f7b0af066f0, min_dup_ls_status_info_:{tenant_id:0, ls_id:{id:-1}, ls_group_id:18446744073709551615, status:"UNKNOWN", unit_group_id:18446744073709551615, primary_zone:"", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, tenant_schema_dup_tablet_set_.size():0, scan_task_execute_interval_:10000000, last_dup_ls_refresh_time_:0, last_dup_schema_refresh_time_:0, last_scan_task_succ_time_:1742176605114860, max_execute_interval_:12989480}) [2025-03-17 09:56:57.128120] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=31] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=29116416) [2025-03-17 09:56:57.128198] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=40] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=79, size_used=0, mem_used=0) [2025-03-17 09:56:57.128251] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=21] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:57.150125] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=47, avg_log_gen_to_freeze_time=2735, avg_log_gen_to_submit_time=2745, avg_log_submit_to_flush_time=12590, avg_log_submit_to_slide_time=12605) [2025-03-17 09:56:57.150224] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=31] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=27206, hit_cnt=47, read_cnt=47, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:57.153657] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=725, get_gts_with_stc_cnt=68, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=69, try_wait_gts_elapse_cnt=0) [2025-03-17 09:56:57.153689] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=30] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1742176617153634], gts:1742176617153351000, latest_srr:[mts=1742176617153634]}) [2025-03-17 09:56:57.153702] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=12] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:56:57.174549] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=29] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:56:57.175936] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=23] PNIO [ratelimit] time: 1742176617175934, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007622, add_bytes: 0 [2025-03-17 09:56:57.178796] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.179753] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539188][T1004_Occam][T1004][YB420A0B093E-00062FB86F654125-0-0] [lt=35] ====== tenant freeze timer task ====== [2025-03-17 09:56:57.180611] INFO [PALF] inner_write_once_ (log_block_handler.cpp:410) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=38] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=1920532, this={dio_aligned_buf:{buf_write_offset:3726, buf_padding_size:370, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:4317724, truncate_used_ts:1456373}, log_block_size:67108864, io_fd:{first_id:-1, second_id:178, third_id:-1, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333a080}}, aligned_buf_len=4096, aligned_block_offset=1916928, buf_len=122, total_write_size=12050, total_write_size_after_dio=417792, ob_pwrite_used_ts=1546579, count=99, each_pwrite_cost=15622) [2025-03-17 09:56:57.180648] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12662, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:56:57.180658] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162412052}], scn_array=[cnt:1, {val:1742176617153351000, v:0}], curr_size=122, accum_size=1220, time_cost=23107) [2025-03-17 09:56:57.180675] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:56:57.180697] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=20, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:56:57.180718] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:56:57.180758] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=9, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:56:57.180771] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12743, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:56:57.180787] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=39, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:56:57.181347] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539162][T1004_Occam][T1004][YB420A0B093E-00062FB7FA8C57BE-0-0] [lt=28] for each ls(succ_num=2, ret=0, ret="OB_SUCCESS") [2025-03-17 09:56:57.181544] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=25] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:56:57.185045] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40326089932, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:57.213892] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539374][T1003_Occam][T1003][YB420A0B093E-00062FB804154126-0-0] [lt=51] ====== tenant freeze timer task ====== [2025-03-17 09:56:57.246416] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=6][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:57.246483] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=66][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:57.246881] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176615651502000, replayable_scn:1742176615651502000, readable_scn:1742176615150772004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176616796101000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:57.246921] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176615651502000, replayable_scn:1742176615651502000, readable_scn:1742176615150772004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176616796101000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:57.246943] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176615651502000, replayable_scn:1742176615651502000, readable_scn:1742176615150772004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176616796101000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:57.247108] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=33] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=1) [2025-03-17 09:56:57.248193] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACF9B-0-0] [lt=13][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:57.248225] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=16][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:56:57.248236] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:56:57.257031] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [118838][T1_L1_G10000][T1][YB420A0B093F-00062FB9C506A177-0-0] [lt=6] get number(ret=0, service_type_=0, range=1, base_id=1742176617256649000, start_id=1742176617256649000, end_id=1742176617256649001) [2025-03-17 09:56:57.260617] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859678216}], scn_array=[cnt:1, {val:1742176617193780000, v:0}], curr_size=588, accum_size=8005, time_cost=66358) [2025-03-17 09:56:57.260788] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78621-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c79f8, ref_cnt_:1, raw_buf_:0x7f7a60358150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78621-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176617238550, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:22190, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:57.260853] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78621-0-0] [lt=0][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.260895] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78621-0-0] [lt=6][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.261253] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78621-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade8c4d40, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222515477, assoc_session_id:3222515477, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617237490, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617232171, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a393e4038, id:3222515477, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade8c4d40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.266106] WDIAG push_back (ob_fixed_array.h:71) [118838][T1_L1_G10000][T1][YB420A0B093F-00062FB9C506A17A-0-0] [lt=21][errcode=-4006] array not init(ret=-4006, capacity=0) [2025-03-17 09:56:57.266135] WDIAG [SQL.ENG] fill_storage_feedback_info (ob_table_scan_op.cpp:1929) [118838][T1_L1_G10000][T1][YB420A0B093F-00062FB9C506A17A-0-0] [lt=28][errcode=0] push back table_id-row_count failed but we won't stop execution(tmp_ret=-4006) [2025-03-17 09:56:57.275555] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=34] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=44, avg_log_batch_cnt=1, total_group_log_size=24467, avg_group_log_size=556) [2025-03-17 09:56:57.276377] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872700029}], scn_array=[cnt:1, {val:1742176617153351001, v:0}], curr_size=122, accum_size=1342, time_cost=18801) [2025-03-17 09:56:57.276456] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=78] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1342) [2025-03-17 09:56:57.276497] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1342) [2025-03-17 09:56:57.276564] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:57.278865] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.281508] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539350][T1003_Occam][T1003][YB420A0B093E-00062FB7FA9C57BE-0-0] [lt=30] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:56:57.286243] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=50][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463501430}], scn_array=[cnt:1, {val:1742176617265185000, v:0}], curr_size=122, accum_size=1539, time_cost=10672) [2025-03-17 09:56:57.294127] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=24] PNIO [ratelimit] time: 1742176617294125, bytes: 2535004989224, bw: 0.911460 MB/s, add_ts: 1001916, add_bytes: 957566 [2025-03-17 09:56:57.307984] INFO [LIB] stat (utility.h:1210) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=34] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:56:57.319620] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A77-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:56:57.319662] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A77-0-0] [lt=40] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:56:57.332062] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859678804}], scn_array=[cnt:1, {val:1742176617193780001, v:0}], curr_size=122, accum_size=8127, time_cost=24031) [2025-03-17 09:56:57.332098] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8127) [2025-03-17 09:56:57.332119] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=19, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:56:57.332175] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=8, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:56:57.332203] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=28] [PALF STAT LOG LIFE TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=26773, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:56:57.332212] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=15, avg_log_gen_to_freeze_time=2626, avg_log_gen_to_submit_time=2638, avg_log_submit_to_flush_time=24119, avg_log_submit_to_slide_time=24135) [2025-03-17 09:56:57.332229] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=37, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:56:57.332238] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8127, hit_cnt=15, read_cnt=15, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:57.340909] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=13] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:57.344450] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [117821][T1_TenantWeakRe][T1][YB420A0B093E-00062FB95C164208-0-0] [lt=26] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=3, total_release_part_ctx_count=152) [2025-03-17 09:56:57.347673] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=31] gc stale ls task succ [2025-03-17 09:56:57.357478] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=17] ====== check clog disk timer task ====== [2025-03-17 09:56:57.357538] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=52] get_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:56:57.376854] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=15][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859678926}], scn_array=[cnt:1, {val:1742176617330182000, v:0}], curr_size=588, accum_size=588, time_cost=44682) [2025-03-17 09:56:57.376979] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=36] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7410, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F4-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176617342182, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:34759, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:57.377033] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=52][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.377054] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=20][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.377072] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=17][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.377080] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7410, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F4-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176617342182, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:34759, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.377110] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=24][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7410, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F4-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176617342182, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:34759, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176617377107}}, part_id:-1}) [2025-03-17 09:56:57.378402] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463501552}], scn_array=[cnt:1, {val:1742176617344056000, v:0}], curr_size=587, accum_size=2126, time_cost=33734) [2025-03-17 09:56:57.378395] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.378442] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=46][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.378449] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a2293d15a, src_buf_size=1611) [2025-03-17 09:56:57.378456] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6378, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F4-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176617377124, enqueue_used:2, dequeue_used:44, submit_used:1, return_used:1205, callback_enqueue_used:1, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.378477] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=18][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6378, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F4-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176617377124, enqueue_used:2, dequeue_used:44, submit_used:1, return_used:1205, callback_enqueue_used:1, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176617378476}}, part_id:-1}) [2025-03-17 09:56:57.378468] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872700151}], scn_array=[cnt:1, {val:1742176617153351002, v:0}], curr_size=122, accum_size=122, time_cost=20816) [2025-03-17 09:56:57.378598] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222502994, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2c4d40}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:57.378645] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:26630}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634290} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:57.378678] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=28] end stmt(ret=0, tx_id=9634290, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2c4d40, tx_id:{txid:9634290}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222502994, assoc_session_id:3222502994, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176617340867, active_ts:1742176617340867, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176647340861, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461456305760, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617321692}], exec_info_reap_ts:1742176617321691, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617321688, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222502994, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2c4d40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.378845] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634290}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176647340861, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617340867}{self_ls_id:{id:1}, session_id:3222502994, associated_session_id:3222502994, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634290}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461456305760, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617341925, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634290} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.378927] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.378920] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=66] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634290}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176647340861, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617340867}{self_ls_id:{id:1}, session_id:3222502994, associated_session_id:3222502994, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634290}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461456305760, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617341925, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634290} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.378963] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=38] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634290}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176617378842, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:57.378979] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=15] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2c4d40, tx_id:{txid:9634290}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222502994, assoc_session_id:3222502994, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176617340867, active_ts:1742176617340867, commit_ts:-1, finish_ts:1742176617378195, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176647340861, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461456305760, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617321692}], exec_info_reap_ts:1742176617321691, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617321688, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:57.379061] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F4-0-0] [lt=1][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:57.380761] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162412296}], scn_array=[cnt:1, {val:1742176617153351002, v:0}], curr_size=122, accum_size=244, time_cost=23074) [2025-03-17 09:56:57.383679] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=17][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:827, tid:3539397}]) [2025-03-17 09:56:57.385362] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40315604172, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:57.415541] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.415864] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.415910] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.415943] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.416101] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.416170] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.416171] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.427883] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859679514}], scn_array=[cnt:1, {val:1742176617394213000, v:0}], curr_size=588, accum_size=1176, time_cost=33251) [2025-03-17 09:56:57.432394] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=29] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:57.444523] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:57.446245] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=17][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.446288] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=41][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.446308] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=19][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.446314] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7ba8, ref_cnt_:1, raw_buf_:0x7f7aa4274150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617445750, enqueue_used:2, dequeue_used:39, submit_used:1, return_used:419, callback_enqueue_used:3, callback_dequeue_used:28, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.446352] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7ba8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617445750, enqueue_used:2, dequeue_used:39, submit_used:1, return_used:419, callback_enqueue_used:3, callback_dequeue_used:28, callback_finish_used:-1, end_used:1742176617446347}}, part_id:-1}) [2025-03-17 09:56:57.446357] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=40][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617445750, enqueue_used:2, dequeue_used:39, submit_used:1, return_used:419, callback_enqueue_used:3, callback_dequeue_used:28, callback_finish_used:132, end_used:-27}}) [2025-03-17 09:56:57.446396] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=38][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:57.446406] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:57.446670] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=22][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.446686] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=15][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.446691] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d00f15a, src_buf_size=1611) [2025-03-17 09:56:57.446696] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7b18, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617446432, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:201, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.446723] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=18][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7b18, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617446432, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:201, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176617446715}}, part_id:-1}) [2025-03-17 09:56:57.446749] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176617446432, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:201, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:73, end_used:-26}}) [2025-03-17 09:56:57.446769] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=20][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:57.446778] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=7][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.446797] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=19][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.446814] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=15][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:57.446829] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=14][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:57.446850] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=21][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:57.446910] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=60][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:57.446917] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:57.446928] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=11][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:57.446941] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=10][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:57.446948] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=7][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176647345606, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:57.447017] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=68][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176647345606, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7a2281c8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: D4346B1A80300600, int: 1742176617444564}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}}) [2025-03-17 09:56:57.447073] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=54][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7a2281c8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: D4346B1A80300600, int: 1742176617444564}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:57.447098] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=24][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:57.447108] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:57.447115] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}, ret=-4016) [2025-03-17 09:56:57.447218] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=26] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=0) [2025-03-17 09:56:57.447181] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=24][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7a2281ce10, local_datums:0x7f7a2281ca60}, datums:[col_id=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7a2281c8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c8f5, hex: D4346B1A80300600, int: 1742176617444564}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c8fd, hex: D4346B1A80300600, int: 1742176617444564}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c905, hex: D4346B1A80300600, int: 1742176617444564}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c90d, hex: 0800000000000000, int: 8}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c915, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7a2281c91d, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2281c923, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 28, flag: 0, null: 0, ptr: 0x7f7a2281c934, hex: 44424D535F53544154532E50555247455F5354415453284E554C4C29, cstr: DBMS_STATS.PURGE_STATS(NULL)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7a2281c950, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c959, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c961, hex: 1227F4050D300600, int: 1741682352858898}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2281c969, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7a2281c97a, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176647345606, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176647345606, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176617394213000, v:0}, tx_id:{txid:9634294}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634294}, tx_desc:{this:0x7f7acf32c060, tx_id:{txid:9634294}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519957, assoc_session_id:3222519957, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176617444266, active_ts:1742176617444266, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176647444261, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461559893798, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176617442070}], exec_info_reap_ts:1742176617442069, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617442066, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:57.447261] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=78][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:57.447279] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=16][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7a2281c8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c8f5, hex: D4346B1A80300600, int: 1742176617444564}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7a2281c8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2281c8f5, hex: D4346B1A80300600, int: 1742176617444564}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:57.444564"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176647345606, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:57.447330] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=50][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:57.447335] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:57.447345] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:57.447353] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=7][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:57.447366] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=9][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:57.447373] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:57.447380] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=7][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:57.447387] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:57.447392] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176647445606, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388355, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf32c060, tx_id:{txid:9634294}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519957, assoc_session_id:3222519957, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176617444266, active_ts:1742176617444266, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176647444261, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461559893798, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176617442070}], exec_info_reap_ts:1742176617442069, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617442066, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634294}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176647345606, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176647345606, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176647345606, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:57.447960] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=567][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:57.447966] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:57.447975] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:57.447979] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:57.447984] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:57.448004] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=18][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:57.448011] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:57.448015] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:57.448024] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=9][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:57.448029] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:57.448033] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:57.448038] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:57.448043] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}) [2025-03-17 09:56:57.448049] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:57.448071] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:57.448090] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=13] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222519957, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf32c060}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:57.448124] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=19] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634294} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:57.448168] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=39] end stmt(ret=0, tx_id=9634294, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf32c060, tx_id:{txid:9634294}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519957, assoc_session_id:3222519957, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176617444266, active_ts:1742176617444266, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176647444261, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461559893798, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617442070}], exec_info_reap_ts:1742176617442069, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617442066, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222519957, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf32c060}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.448221] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=52][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:57.448227] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=5][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:57.448230] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=3][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:57.448240] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=4][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:57.448255] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=13][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:57.448271] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=15][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')) [2025-03-17 09:56:57.448315] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=35][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:57.448326] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=10][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), is_user_sql=false) [2025-03-17 09:56:57.448345] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=19][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176617444564), usec_to_time(1742176617444564), 8, usec_to_time(1742176617444564), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), conn=0x7f7ac70b2d90) [2025-03-17 09:56:57.448356] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=10][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:57.448431] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=8][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634294}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176647444261, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617445331}{self_ls_id:{id:1}, session_id:3222519957, associated_session_id:3222519957, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634294}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461559893798, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617445331, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634294} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.448512] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=67] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634294}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176647444261, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617445331}{self_ls_id:{id:1}, session_id:3222519957, associated_session_id:3222519957, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634294}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461559893798, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617445331, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634294} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.448575] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=57] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634294}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176617448426, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:57.448595] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=17] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf32c060, tx_id:{txid:9634294}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519957, assoc_session_id:3222519957, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176617444266, active_ts:1742176617444266, commit_ts:-1, finish_ts:1742176617448524, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176647444261, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461559893798, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617442070}], exec_info_reap_ts:1742176617442069, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617442066, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:57.448697] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=44][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:57.448727] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=30][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:57.448735] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F5-0-0] [lt=6][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:57.465437] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=29] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}]) [2025-03-17 09:56:57.469802] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=1][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.469838] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=35][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.469857] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.469863] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c71d0, ref_cnt_:1, raw_buf_:0x7f7ac4e04150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB72EE5F3A0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469325, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:429, callback_enqueue_used:3, callback_dequeue_used:23, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:57.469903] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=29][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c71d0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB72EE5F3A0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469325, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:429, callback_enqueue_used:3, callback_dequeue_used:23, callback_finish_used:-1, end_used:1742176617469894}}, part_id:-1}) [2025-03-17 09:56:57.469919] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=0][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469325, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:429, callback_enqueue_used:3, callback_dequeue_used:23, callback_finish_used:123, end_used:-28}}) [2025-03-17 09:56:57.469957] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=34][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:57.469969] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=10][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:57.470254] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=19][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.470292] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=37][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.470301] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=9][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a3f2b3510, src_buf_size=295) [2025-03-17 09:56:57.470312] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7338, ref_cnt_:1, raw_buf_:0x7f7a3f2b2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB72EE5F3A0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469980, enqueue_used:0, dequeue_used:12, submit_used:0, return_used:244, callback_enqueue_used:0, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:57.470370] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=47][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7338, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB72EE5F3A0-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469980, enqueue_used:0, dequeue_used:12, submit_used:0, return_used:244, callback_enqueue_used:0, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176617470360}}, part_id:-1}) [2025-03-17 09:56:57.470382] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176617469980, enqueue_used:0, dequeue_used:12, submit_used:0, return_used:244, callback_enqueue_used:0, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176617470360}}) [2025-03-17 09:56:57.470410] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=27][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:57.470425] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=13][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aaacbe080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.470449] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=23][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aaacbe080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.470471] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=20][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:57.470489] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=17][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:57.470516] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=26][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f78178262f0, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f7a8d8839e0, last_rowkey:0x7f7a8d883b48, discrete_rowkey_array:0x7f7a8d883c68, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883a70, hex: EB03000000000000, int: 1003}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883aa8, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b08, hex: C8F2AD7DB79BD4E7, int: -1741595943495667000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b20, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f7a8d883963, ps_node_array:0x7f7a8d8839a0, data_buf:0x7f7a8d883963, data_buf_size:1031}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:57.470581] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=64][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=2, prefetched_cnt=0) [2025-03-17 09:56:57.470588] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=6][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:57.470595] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=6][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:57.470612] WDIAG [STORAGE] inner_open (ob_sstable_multi_version_row_iterator.cpp:162) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=12][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:57.470620] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=8][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176647367290, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc5e050, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176617344056000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7acacd2090, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176617467634, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:57.471030] WDIAG [STORAGE] scan (ob_sstable.cpp:427) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=408][errcode=-4016] Fail to open row scanner(ret=-4016, param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176647367290, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc5e050, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176617344056000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7acacd2090, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176617467634, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}, key_range={start_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814610, hex: E903000000000000, int: 1001}, idx=1:MIN_OBJ], store_rowkey:{"BIGINT":1001},MIN}, end_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814690, hex: E903000000000000, int: 1001}, idx=1:MAX_OBJ], store_rowkey:{"BIGINT":1001},MAX}, group_idx:0, border_flag:{flag:0}}, *this={ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2025-03-17 09:56:57.471506] WDIAG [STORAGE] construct_iters (ob_multiple_scan_merge.cpp:198) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=474][errcode=-4016] Fail to get iterator(ret=-4016, i=1, *iter_pram={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}) [2025-03-17 09:56:57.471855] WDIAG [STORAGE] open (ob_multiple_scan_merge.cpp:60) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=349][errcode=-4016] fail to construct iters(ret=-4016) [2025-03-17 09:56:57.471863] WDIAG [STORAGE] init_and_open_scan_merge_iter_ (ob_table_scan_iterator.cpp:651) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7][errcode=-4016] Fail to open multiple merge iterator(ret=-4016) [2025-03-17 09:56:57.471871] WDIAG [STORAGE] open_iter (ob_table_scan_iterator.cpp:558) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7][errcode=-4016] init and open scan merge iterator failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:57.471880] WDIAG [STORAGE] init (ob_table_scan_iterator.cpp:355) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=8][errcode=-4016] fail to open iter(ret=-4016, cached_iter_node_=NULL, *this={}) [2025-03-17 09:56:57.471890] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2373) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=9][errcode=-4016] failed to init table scan iterator, (ret=-4016) [2025-03-17 09:56:57.471897] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2379) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=6][errcode=-4016] failed to do table scan(ret=-4016, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1001},MIN;{"BIGINT":1001},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176647367290, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176617467634, need_update_tablet_param:false}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2025-03-17 09:56:57.472195] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:674) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=297][errcode=-4016] failed to do table scan(ret=-4016, &iter=0x7f7b0cc5d3d0, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1001},MIN;{"BIGINT":1001},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176647367290, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176617467634, need_update_tablet_param:false}) [2025-03-17 09:56:57.472406] WDIAG [STORAGE] table_scan (ob_access_service.cpp:371) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=210][errcode=-4016] Fail to scan table, (ret=-4016, ls_id={id:1}, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1001},MIN;{"BIGINT":1001},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176647367290, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176617467634, need_update_tablet_param:false}) [2025-03-17 09:56:57.472633] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_iter.cpp:83) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=226][errcode=-4016] fail to scan table(scan_param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1001},MIN;{"BIGINT":1001},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176647367290, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176617467634, need_update_tablet_param:false}, ret=-4016) [2025-03-17 09:56:57.472865] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:569) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=231][errcode=-4016] iter tree failed to do table scan(ret=-4016) [2025-03-17 09:56:57.472871] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=5][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:57.472875] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:102) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=3][errcode=-4016] start das task failed(ret=-4016, *task_wrapper.at(i)={parent:{tenant_id:1, task_id:212095022, op_type:1, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7ace816d00, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222493517, assoc_session_id:3222493517, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617468697, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617467634, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, snapshot:{this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, tablet_id:{id:345}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:345}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:345, related_table_ids:[cnt:0], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:1}, in_retry:0, partition_id:345, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:, plan_line_id:0}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:345, access_column_ids:[cnt:13, 18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1741278087142176, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, pd_expr_spec:{calc_exprs:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], access_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:16, pushdown_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}], is_get:false, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0, rowkey_exprs:[cnt:0], table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, doc_id_idx:-1, vec_vid_idx:-1, index_merge_idx:-1, pre_query_range:[], is_index_merge:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:-1, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1742176647367290, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tsc_monitor_info:{*io_read_bytes_:295*ssstore_read_bytes_:0*ssstore_read_row_cnt_:0*memstore_read_row_cnt_:0}, key_ranges:[], ss_key_ranges:[], mbr_filters:[], scan_op_id:18446744073709551615, scan_rows_size:-1, das_tasks_key:{fake_unique_id_:0, timestamp_:0}}, scan_range:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1001},MIN;{"BIGINT":1001},MAX)"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}}) [2025-03-17 09:56:57.473193] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=317][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:57.473203] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=9][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:57.473207] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:260) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=3][errcode=-4016] failed to execute all das task(ret=-4016) [2025-03-17 09:56:57.473211] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2699) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=4][errcode=-4016] execute all das scan task failed(ret=-4016) [2025-03-17 09:56:57.473215] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1822) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=4][errcode=-4016] fail to do table scan(ret=-4016) [2025-03-17 09:56:57.473220] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2588) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=4][errcode=-4016] failed to init before get row(ret=-4016) [2025-03-17 09:56:57.473226] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=6][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:56:57.473234] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f781780bfb0}, id=0, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:56:57.473244] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=9][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:56:57.473251] WDIAG [SQL.EXE] sync_send_result (ob_remote_executor_processor.cpp:286) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:57.473258] WDIAG [SQL.EXE] execute_remote_plan (ob_remote_executor_processor.cpp:485) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7][errcode=-4016] sync send result failed(ret=-4016) [2025-03-17 09:56:57.473281] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=7] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace816d00, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222493517, assoc_session_id:3222493517, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617468697, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617467634, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ad1c7b118, id:3222493517, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace816d00}, exec_ctx.get_errcode()=0) [2025-03-17 09:56:57.473318] WDIAG [SQL.EXE] execute_with_sql (ob_remote_executor_processor.cpp:716) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB72EE5F3A0-0-0] [lt=31][errcode=-4016] execute remote plan failed(ret=-4016, task={tenant_schema_version:1741661297536408, sys_schema_version:1741661297536408, runner_svr:"192.168.1.62:2882", ctrl_svr:"192.168.1.63:2882", task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.63:2882", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:18446744073709551615}, task_id:0, task_cnt:0}, remote_sql_info:{use_ps:true, is_batched_stmt:false, is_original_ps_mode:false, ps_param_cnt:2, remote_sql:"SELECT * FROM __all_ls_status where tenant_id = ? and flag like ?", ps_params:[{obj:{"BIGINT":1001}, accuracy:{length:4, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}, {obj:{"VARCHAR":"%DUPLICATE%", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}, accuracy:{length:11, precision:1, scale:-1}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}}]}, snapshot:{this:0x7f7817804208, valid:false, source:0, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}}, exec_ctx_.get_das_ctx().get_snapshot()={this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}) [2025-03-17 09:56:57.479000] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=15] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:56:57.479035] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=34] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:56:57.479045] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.489079] WDIAG [SHARE] get_source (ob_log_restore_source_mgr.cpp:195) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=32][errcode=-4018] table_operator_ get_source failed(ret=-4018) [2025-03-17 09:56:57.490429] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=39][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.490477] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=46][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.490495] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.490507] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6888, ref_cnt_:1, raw_buf_:0x7f7a3f2b2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0071-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617489994, enqueue_used:2, dequeue_used:20, submit_used:1, return_used:389, callback_enqueue_used:3, callback_dequeue_used:17, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:57.490549] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=35][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6888, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0071-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617489994, enqueue_used:2, dequeue_used:20, submit_used:1, return_used:389, callback_enqueue_used:3, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176617490545}}, part_id:-1}) [2025-03-17 09:56:57.490554] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=19][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617489994, enqueue_used:2, dequeue_used:20, submit_used:1, return_used:389, callback_enqueue_used:3, callback_dequeue_used:17, callback_finish_used:149, end_used:-30}}) [2025-03-17 09:56:57.490593] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=34][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:57.490608] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=12][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:57.490942] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=25][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.490975] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=33][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.490982] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a96a17510, src_buf_size=295) [2025-03-17 09:56:57.490989] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6450, ref_cnt_:1, raw_buf_:0x7f7a96a16150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0071-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617490619, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:292, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:57.491022] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=27][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6450, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0071-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617490619, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:292, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176617491017}}, part_id:-1}) [2025-03-17 09:56:57.491026] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176617490619, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:292, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:104, end_used:-26}}) [2025-03-17 09:56:57.491070] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=40][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:57.491084] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=12][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac25a2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.491112] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=28][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac25a2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:57.491136] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=21][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:57.491164] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=27][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc5f450}) [2025-03-17 09:56:57.491190] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=25][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc601f8, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:0, rowkey_end_idx:0, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc601f8, micro_handle:NULL}) [2025-03-17 09:56:57.491240] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=49][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:56:57.491247] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=6][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac4e0ea58, local_datums:0x7f7ac4e0ea58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:56:57.491459] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=211][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac4e0ea58, local_datums:0x7f7ac4e0ea58}, datums:[]}) [2025-03-17 09:56:57.491475] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=15][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176617344056000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:56:57.491485] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=10][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7ac4e0dac0) [2025-03-17 09:56:57.491494] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=9][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:56:57.491503] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1003},{"BIGINT":1};{"BIGINT":1003},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176647389121, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7ac4e0d3d0, valid:true, source:2, core:{version:{val:1742176617344056000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176617477149, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7ac4e099a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7ac4e0d6a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7ac4e09848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7ac4e0dac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:56:57.491875] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=371][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:57.491881] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=6][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:56:57.491887] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=6][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:56:57.491893] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=5][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:56:57.491898] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=6][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:56:57.491903] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=5][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:56:57.491909] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=5][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7ac4e05b50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:56:57.491923] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=13][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:56:57.491928] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=5][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:56:57.491936] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:56:57.491944] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:56:57.491952] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7ac4e05b50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:56:57.491962] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=9][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:56:57.491969] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=7][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:57.491978] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:56:57.491987] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=8][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}) [2025-03-17 09:56:57.491997] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=10][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.492025] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0071-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace85fa80, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519428, assoc_session_id:3222519428, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617489913, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617477149, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222519428, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace85fa80}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.495448] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE18-0-0] [lt=21][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.495486] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE18-0-0] [lt=37][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.495529] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE18-0-0] [lt=33][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.495875] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE18-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6a5b9c0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222515666, assoc_session_id:3222515666, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176617494166, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176617494165, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ad1c721f8, id:3222515666, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6a5b9c0}, exec_ctx.get_errcode()=0) [2025-03-17 09:56:57.501058] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=2596, pos=313) [2025-03-17 09:56:57.501605] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=19][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=320, pos=287) [2025-03-17 09:56:57.502242] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=180, pos=160) [2025-03-17 09:56:57.502810] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=394, pos=286) [2025-03-17 09:56:57.503396] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=188, pos=169) [2025-03-17 09:56:57.504097] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570B-0-0] [lt=12][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=10, len=124, pos=107) [2025-03-17 09:56:57.504922] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570C-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11004, add_timestamp:1742176617504905}) [2025-03-17 09:56:57.504945] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3183-0-0] [lt=20] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11004, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:57.505162] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3183-0-0] [lt=21] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11004, locations=[{table_id:11004, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.505196] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3183-0-0] [lt=33] success to process renew task(task={tenant_id:1, table_id:11004, add_timestamp:1742176617504905}, locations=[{table_id:11004, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}, {table_id:11004, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617505135, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.510948] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:57.511018] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=12] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:57.528903] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=39] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=23653, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:56:57.528938] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859680102}], scn_array=[cnt:1, {val:1742176617394213001, v:0}], curr_size=122, accum_size=1298, time_cost=20762) [2025-03-17 09:56:57.528981] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8127) [2025-03-17 09:56:57.530000] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=47] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:56:57.530030] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=21] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=11530) [2025-03-17 09:56:57.538100] INFO [STORAGE.TRANS] generate_weak_read_timestamp_ (ob_ls_wrs_handler.cpp:194) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=18] get wrs ts(ls_id={id:1}, delta=384609, log_service_ts=1742176617153351, min_tx_service_ts=4611686018427387, timestamp={val:1742176617153351002, v:0}) [2025-03-17 09:56:57.538137] INFO [STORAGE.TRANS] print_stat_info (ob_keep_alive_ls_handler.cpp:226) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=33] [Keep Alive Stat] LS Keep Alive Info(tenant_id=1004, LS_ID={id:1}, Not_Master_Cnt=0, Near_To_GTS_Cnt=3, Other_Error_Cnt=0, Submit_Succ_Cnt=97, last_scn="{val:1742176615150772002, v:0}", last_lsn={lsn:872696541}, last_gts={val:1742176617153351000, v:0}, min_start_scn="{val:0, v:0}", min_start_status=1, sys_ls_end_scn={val:0, v:0}) [2025-03-17 09:56:57.548845] INFO [PALF] is_allowed_broadcast_ (palf_handle_impl.h:1404) [3539358][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=78] keep in silent because of disconnected with RS, do not solicit votes.(is_in_silent=false, palf_id=1, is_allowed_broadcast=true) [2025-03-17 09:56:57.562586] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:57.562648] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:57.579075] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.581617] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872700395}], scn_array=[cnt:1, {val:1742176617153351004, v:0}], curr_size=122, accum_size=366, time_cost=23805) [2025-03-17 09:56:57.581588] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463502261}], scn_array=[cnt:1, {val:1742176617544371000, v:0}], curr_size=586, accum_size=2834, time_cost=36641) [2025-03-17 09:56:57.581832] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1742176617344056000, v:0}, new_max_version={val:1742176617344056000, v:0}, last_min_version={val:1742176617143738000, v:0}, last_max_version={val:1742176617143738000, v:0}, total_time_us=38039, delta_us=237775) [2025-03-17 09:56:57.581864] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=29] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1742176617344056000, v:0}, min_version={val:1742176617344056000, v:0}, max_version={val:1742176617344056000, v:0}, in_service=true, leader_epoch=4) [2025-03-17 09:56:57.582917] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162412540}], scn_array=[cnt:1, {val:1742176617153351004, v:0}], curr_size=122, accum_size=488, time_cost=25101) [2025-03-17 09:56:57.582976] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=9, avg_log_submit_to_flush_time=12603, avg_log_submit_to_slide_time=12617) [2025-03-17 09:56:57.583011] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:57.585666] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=26] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40311409868, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:57.588286] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [3539166][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=26] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}]) [2025-03-17 09:56:57.594799] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=31] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2025-03-17 09:56:57.594835] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=33] start lib cache evict(tenant_id=1, mem_hold=283115520, mem_limit=322122545, cache_obj_num=776, cache_node_num=773) [2025-03-17 09:56:57.594850] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=13] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=283115520, mem_limit=322122545, cache_obj_num=776, cache_node_num=773) [2025-03-17 09:56:57.594865] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=14] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:57.597014] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=6] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:57.598722] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.598762] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=39][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.598784] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.598795] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c69a8, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F6-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176617598194, enqueue_used:1, dequeue_used:48, submit_used:1, return_used:370, callback_enqueue_used:3, callback_dequeue_used:102, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.598848] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=45][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c69a8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F6-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176617598194, enqueue_used:1, dequeue_used:48, submit_used:1, return_used:370, callback_enqueue_used:3, callback_dequeue_used:102, callback_finish_used:-1, end_used:1742176617598844}}, part_id:-1}) [2025-03-17 09:56:57.598862] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=35][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:57.599154] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=25][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.599187] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=32][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.599195] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=8][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79f029915a, src_buf_size=1611) [2025-03-17 09:56:57.599204] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7140, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F6-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176617598912, enqueue_used:1, dequeue_used:34, submit_used:1, return_used:175, callback_enqueue_used:2, callback_dequeue_used:17, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.599242] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=34][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7140, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F6-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176617598912, enqueue_used:1, dequeue_used:34, submit_used:1, return_used:175, callback_enqueue_used:2, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176617599240}}, part_id:-1}) [2025-03-17 09:56:57.599359] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222515551, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf20cb20}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:57.599414] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634297} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:57.599475] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=56] end stmt(ret=0, tx_id=9634297, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf20cb20, tx_id:{txid:9634297}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222515551, assoc_session_id:3222515551, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176617596698, active_ts:1742176617596698, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176647596692, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461712354428, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617593502}], exec_info_reap_ts:1742176617593501, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617593498, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222515551, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf20cb20}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.598400] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=10] Dumping All Cache Objs(alloc_obj_list.count()=776, alloc_obj_list=[{obj_id:51, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196394}, {obj_id:72, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180596}, {obj_id:74, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188388}, {obj_id:139, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160477}, {obj_id:329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:327911}, {obj_id:381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189321}, {obj_id:399, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178578}, {obj_id:410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179268}, {obj_id:447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194846}, {obj_id:495, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196117}, {obj_id:50913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177580}, {obj_id:53754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176496}, {obj_id:54672, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233845}, {obj_id:54692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235254}, {obj_id:6335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185047}, {obj_id:6380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190234}, {obj_id:57035, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276727}, {obj_id:57055, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184180}, {obj_id:57801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194188}, {obj_id:58087, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180124}, {obj_id:58344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199622}, {obj_id:59240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187102}, {obj_id:59241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:273946}, {obj_id:59242, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197398}, {obj_id:59249, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280549}, {obj_id:59255, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196978}, {obj_id:59257, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:288566}, {obj_id:59259, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:334006}, {obj_id:59260, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175776}, {obj_id:59261, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:317353}, {obj_id:59304, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195857}, {obj_id:59875, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177585}, {obj_id:59876, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177622}, {obj_id:59879, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178040}, {obj_id:59881, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235344}, {obj_id:59882, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187939}, {obj_id:59884, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183047}, {obj_id:59890, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178857}, {obj_id:59895, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194979}, {obj_id:59897, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235364}, {obj_id:59900, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199510}, {obj_id:59905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209961}, {obj_id:59907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195635}, {obj_id:59908, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59917, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200616}, {obj_id:59918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195188}, {obj_id:59919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179136}, {obj_id:59920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177462}, {obj_id:59921, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188401}, {obj_id:59922, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193417}, {obj_id:59923, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59924, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178328}, {obj_id:59925, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59926, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182265}, {obj_id:59927, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192859}, {obj_id:59928, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59929, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248320}, {obj_id:59930, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59931, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185648}, {obj_id:59932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213471}, {obj_id:59935, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194647}, {obj_id:59936, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177264}, {obj_id:59937, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185959}, {obj_id:59938, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248440}, {obj_id:59939, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226192}, {obj_id:59940, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176800}, {obj_id:59941, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174944}, {obj_id:59942, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177893}, {obj_id:59943, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194037}, {obj_id:59944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181347}, {obj_id:59945, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185263}, {obj_id:59946, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185681}, {obj_id:59947, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196901}, {obj_id:59948, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203466}, {obj_id:59949, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193243}, {obj_id:59950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189090}, {obj_id:59951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190242}, {obj_id:59952, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194564}, {obj_id:59958, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59959, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185063}, {obj_id:59960, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59962, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59971, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:59972, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:252561}, {obj_id:59978, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:59979, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176224}, {obj_id:59981, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186299}, {obj_id:59982, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179554}, {obj_id:59984, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200201}, {obj_id:59985, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213559}, {obj_id:59986, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257040}, {obj_id:59988, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:59989, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179720}, {obj_id:59998, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207279}, {obj_id:60000, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278740}, {obj_id:60001, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183087}, {obj_id:60002, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267931}, {obj_id:60003, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251136}, {obj_id:60005, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:324244}, {obj_id:60007, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:60008, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:492243}, {obj_id:60009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60010, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:60011, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:60012, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:60015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179886}, {obj_id:60016, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:60017, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181705}, {obj_id:60018, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:60022, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178376}, {obj_id:60023, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:60024, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178376}, {obj_id:60025, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:60026, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:60027, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275212}, {obj_id:60028, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249175}, {obj_id:60029, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185679}, {obj_id:60052, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:60053, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177800}, {obj_id:60054, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204741}, {obj_id:60056, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215059}, {obj_id:60061, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180166}, {obj_id:60064, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195605}, {obj_id:60067, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204726}, {obj_id:60069, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240092}, {obj_id:60078, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249595}, {obj_id:60080, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193936}, {obj_id:60332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203439}, {obj_id:60333, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60424, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200032}, {obj_id:60442, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197358}, {obj_id:60660, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180250}, {obj_id:60662, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180358}, {obj_id:60673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192624}, {obj_id:60674, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190259}, {obj_id:60686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192062}, {obj_id:60688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192617}, {obj_id:60690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196328}, {obj_id:60691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205892}, {obj_id:60692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185222}, {obj_id:60743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204639}, {obj_id:60980, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233446}, {obj_id:61026, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200592}, {obj_id:61082, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185236}, {obj_id:61083, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222880}, {obj_id:61084, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180498}, {obj_id:61089, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181704}, {obj_id:61091, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206237}, {obj_id:61092, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184745}, {obj_id:61093, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183368}, {obj_id:61123, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197825}, {obj_id:61124, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181456}, {obj_id:61125, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189172}, {obj_id:61126, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61128, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61129, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61130, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61132, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61134, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224140}, {obj_id:61135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183490}, {obj_id:61176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196918}, {obj_id:61181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202039}, {obj_id:61182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:61183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180267}, {obj_id:61184, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176208}, {obj_id:61185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193061}, {obj_id:61192, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199220}, {obj_id:61193, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182816}, {obj_id:61194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187579}, {obj_id:61195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225220}, {obj_id:61197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180400}, {obj_id:61198, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182234}, {obj_id:61199, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176192}, {obj_id:61200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178808}, {obj_id:61201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178464}, {obj_id:61202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194564}, {obj_id:61206, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180187}, {obj_id:61207, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184050}, {obj_id:61208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179059}, {obj_id:61209, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179107}, {obj_id:61210, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183376}, {obj_id:61212, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184262}, {obj_id:61213, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177856}, {obj_id:61214, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178608}, {obj_id:61216, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182348}, {obj_id:61217, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181360}, {obj_id:61218, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177688}, {obj_id:61219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175744}, {obj_id:61221, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198173}, {obj_id:61233, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178568}, {obj_id:61234, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178568}, {obj_id:61238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176496}, {obj_id:61239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195916}, {obj_id:61240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191698}, {obj_id:61241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61242, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203003}, {obj_id:61243, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179032}, {obj_id:61244, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179032}, {obj_id:61245, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61246, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202624}, {obj_id:61247, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203148}, {obj_id:61248, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238146}, {obj_id:61249, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240110}, {obj_id:61250, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177088}, {obj_id:61251, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176763}, {obj_id:61252, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177200}, {obj_id:61253, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:61254, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:61255, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180574}, {obj_id:61256, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185347}, {obj_id:61257, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187858}, {obj_id:61258, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204326}, {obj_id:61259, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223580}, {obj_id:61260, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197140}, {obj_id:61261, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185593}, {obj_id:61263, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178739}, {obj_id:61264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207006}, {obj_id:61265, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237389}, {obj_id:61266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189942}, {obj_id:61267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238589}, {obj_id:61268, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214079}, {obj_id:61269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:61270, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:61271, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209930}, {obj_id:61272, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187946}, {obj_id:61273, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236433}, {obj_id:61274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199161}, {obj_id:61275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177864}, {obj_id:61276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178589}, {obj_id:61277, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201976}, {obj_id:61278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:273840}, {obj_id:61279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186702}, {obj_id:61280, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184321}, {obj_id:61281, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182076}, {obj_id:61282, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182688}, {obj_id:61283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204022}, {obj_id:61284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176448}, {obj_id:61285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217616}, {obj_id:61286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224439}, {obj_id:61287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192995}, {obj_id:61288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240962}, {obj_id:61289, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182421}, {obj_id:61290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223189}, {obj_id:61291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198587}, {obj_id:61292, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203764}, {obj_id:61293, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211287}, {obj_id:61294, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182312}, {obj_id:61295, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:61296, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:61297, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199853}, {obj_id:61298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175776}, {obj_id:61299, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211964}, {obj_id:61300, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201388}, {obj_id:61301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:61302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211967}, {obj_id:61303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240913}, {obj_id:61304, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206969}, {obj_id:61305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195404}, {obj_id:61306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208488}, {obj_id:61308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205131}, {obj_id:61309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196241}, {obj_id:61310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199179}, {obj_id:61312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207224}, {obj_id:61313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192816}, {obj_id:61314, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184947}, {obj_id:61315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219225}, {obj_id:61316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:494025}, {obj_id:61317, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180312}, {obj_id:61318, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:486704}, {obj_id:61319, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:485788}, {obj_id:61320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1080567}, {obj_id:61321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:61322, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176224}, {obj_id:61323, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182929}, {obj_id:61324, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:61326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:61328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182195}, {obj_id:61329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182404}, {obj_id:61331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:61332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:61333, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180908}, {obj_id:61334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181123}, {obj_id:61335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:61336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:61337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:61338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:61339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:61340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:61341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290643}, {obj_id:61342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230505}, {obj_id:61343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300745}, {obj_id:61344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243566}, {obj_id:61345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210085}, {obj_id:61346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221070}, {obj_id:61347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208917}, {obj_id:61348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207763}, {obj_id:61364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186121}, {obj_id:61365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381025}, {obj_id:61366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217225}, {obj_id:61367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202386}, {obj_id:61368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61369, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216347}, {obj_id:61371, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204192}, {obj_id:61372, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176528}, {obj_id:61373, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200801}, {obj_id:61374, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203691}, {obj_id:61376, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61377, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61379, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202155}, {obj_id:61382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61384, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61386, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205810}, {obj_id:61387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258426}, {obj_id:61388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236555}, {obj_id:61389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178496}, {obj_id:61390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178248}, {obj_id:61391, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215450}, {obj_id:61392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206477}, {obj_id:61393, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201310}, {obj_id:61394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210377}, {obj_id:61395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61396, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61398, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61399, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61400, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180200}, {obj_id:61401, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61402, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61403, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213211}, {obj_id:61404, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210945}, {obj_id:61405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:61406, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212648}, {obj_id:61407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:61408, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254425}, {obj_id:61409, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235203}, {obj_id:61410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209121}, {obj_id:61412, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61413, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211805}, {obj_id:61414, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61415, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61416, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205861}, {obj_id:61417, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61418, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61419, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61420, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61421, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211504}, {obj_id:61422, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61423, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61424, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61425, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61426, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61427, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219069}, {obj_id:61428, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61429, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61430, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217282}, {obj_id:61431, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194233}, {obj_id:61432, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198403}, {obj_id:61433, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196768}, {obj_id:61434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61436, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212753}, {obj_id:61438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213756}, {obj_id:61439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61441, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61442, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61443, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61444, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214979}, {obj_id:61446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185353}, {obj_id:61447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213332}, {obj_id:61448, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61449, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61453, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216791}, {obj_id:61454, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61461, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61463, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61464, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61465, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61466, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218192}, {obj_id:61467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61469, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61476, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61478, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61479, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61481, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:256432}, {obj_id:61483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61484, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61485, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, [2025-03-17 09:56:57.599605] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=1202] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:57.599633] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634297}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176647596692, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617597773}{self_ls_id:{id:1}, session_id:3222515551, associated_session_id:3222515551, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634297}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461712354428, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617597773, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634297} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.599719] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=71] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634297}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176647596692, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617597773}{self_ls_id:{id:1}, session_id:3222515551, associated_session_id:3222515551, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634297}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461712354428, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617597773, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634297} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.599779] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=52] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634297}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176617599628, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:57.599799] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=17] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf20cb20, tx_id:{txid:9634297}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222515551, assoc_session_id:3222515551, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176617596698, active_ts:1742176617596698, commit_ts:-1, finish_ts:1742176617598840, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176647596692, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461712354428, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617593502}], exec_info_reap_ts:1742176617593501, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617593498, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:57.599901] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F6-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:57.604942] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:929, free_obj_hold_cnt:159, allocator used:4186624, allocator total:4439040}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:149, free_obj_hold_cnt:931, allocator used:70838280, allocator total:70838280}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():0, full allocator used:0, full allocator total:0}, tablets_mem=75277320, tablets_mem_limit=1288490180, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:33, free_obj_hold_cnt:50, allocator used:169984, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=1078) [2025-03-17 09:56:57.610682] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859680812}], scn_array=[cnt:1, {val:1742176617593499000, v:0}], curr_size=588, accum_size=2474, time_cost=16476) [2025-03-17 09:56:57.628347] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=22] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=18717696) [2025-03-17 09:56:57.628410] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=25] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:57.635103] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.635178] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.635689] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.635698] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=26] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.635730] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.635762] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.636172] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.636203] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.636537] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.636549] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:57.638768] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570D-0-0] [lt=23] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11031, add_timestamp:1742176617638750}) [2025-03-17 09:56:57.638795] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3184-0-0] [lt=19] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11031, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:57.639088] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3184-0-0] [lt=23] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11031, locations=[{table_id:11031, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.639121] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3184-0-0] [lt=33] success to process renew task(task={tenant_id:1, table_id:11031, add_timestamp:1742176617638750}, locations=[{table_id:11031, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}, {table_id:11031, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176617639074, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:57.641340] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=20] gc stale ls task succ [2025-03-17 09:56:57.651304] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:728) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=20] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=11900000, replace_num=100000) [2025-03-17 09:56:57.651351] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=46] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=10) [2025-03-17 09:56:57.653460] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=23] ====== check clog disk timer task ====== [2025-03-17 09:56:57.653501] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=36] get_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:56:57.655386] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=46] ====== traversal_flush timer task ====== [2025-03-17 09:56:57.655515] INFO [MDS] flush (mds_table_mgr.cpp:195) [117782][T1_TimerWK1][T1][YB420A0B093E-00062FB91AB8CF5F-0-0] [lt=22] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742067170807625001, v:0}}, }, max_consequent_callbacked_scn={val:1742176617544371000, v:0}, *this={this:0x7f7ac1de7150, is_inited:true, freezing_scn:{val:1741629603822325000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176626588679000, latest_log_ts:{val:1742176609930802001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.655588] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [117782][T1_TimerWK1][T1][YB420A0B093E-00062FB91AB8CF5F-0-0] [lt=69] succeed to traversal_flush(ret=0, ls_cnt=1) [2025-03-17 09:56:57.657870] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:57.657906] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:57.657938] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:57.657953] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:57.658625] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:146) [3539201][T1004_TntTransf][T1004][YB420A0B093E-00062FB7FC1623F0-0-0] [lt=30] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=1) [2025-03-17 09:56:57.659537] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=64][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463502847}], scn_array=[cnt:1, {val:1742176617638289000, v:0}], curr_size=817, accum_size=3651, time_cost=20917) [2025-03-17 09:56:57.671988] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=6] ====== checkpoint timer task ====== [2025-03-17 09:56:57.672060] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=42] get_max_applied_scn(scn={val:1742176617638289000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:4, palf_committed_end_lsn_:{lsn:5463503664}, palf_committed_end_scn_:{val:1742176617638289000, v:0}, last_check_scn_:{val:1742176617661790000, v:0}, max_applied_cb_scn_:{val:1742176617638289000, v:0}}) [2025-03-17 09:56:57.672088] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1042) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=22] get_min_unreplayed_log_info(lsn={lsn:64058503}, scn={val:1741318426647365001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741318427097633, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:64058503}, next_to_submit_scn_:{val:1741318426647365001, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741314277197622000, v:0}, iterator_:{iterator_impl:{buf_:0x7f7aef805000, next_round_pread_size:3678208, curr_read_pos:1011, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1011, log_storage_:{start_lsn:{lsn:64057492}, end_lsn:{lsn:64058503}, read_buf:{buf_len_:2932736, buf_:0x7f7aef805000}, block_size:67104768, log_storage_:0x7f7adde63a70, log_storage_:{log_tail:{lsn:5463503664}, readable_log_tail:{lsn:5463503664}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:5435486208}, min_scn:{val:1742171570508885001, v:0}, curr_block_id:81, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1/1/log", dir_fd:265, min_block_id:0, max_block_id:82, curr_writable_block_id:81}, logical_block_size_:67104768, curr_block_writable_size_:39087312, block_header_serialize_buf_:0x7f7adde64738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1741318426647365000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:2, log_size:923, scn_:{val:1741318426647365000, v:0}, data_checksum:7529717, flag:64521}}, init_mode_version:0, accumulate_checksum:983882438, curr_entry_is_padding:0, padding_entry_size:955, padding_entry_scn:{val:1741318426647365000, v:0}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:8104, miss_cnt:0, cache_read_size:64058503, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:56:57.673190] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=108] succ to get rec scn(*this={this:0x7f7ac9e04030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1741318426647365001, v:0}, max_applied_start_working_ts_:{val:1741318426647365001, v:0}}, total_tx_ctx_count:1, active_tx_count:1, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741318291584325, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1078}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.673242] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=51] tx ctx memtable get rec scn(this={ObITable:{this:0x7f7ac579a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1741629604055327, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f7ac579a080, snapshot_version:{val:1741629604055327, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:1742176617661790000, v:0}) [2025-03-17 09:56:57.673384] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=29] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742067170807625001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742067170807625001, v:0}}, }, *this={this:0x7f7ac1de7150, is_inited:true, freezing_scn:{val:1741629603822325000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176626588679000, latest_log_ts:{val:1742176609930802001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.673488] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=102] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7f7ac1df4ad0}, min_rec_scn={val:1742067167143792000, v:0}, ls_id_={id:1}) [2025-03-17 09:56:57.673506] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:0, v:0}) [2025-03-17 09:56:57.673522] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=14] get rec log scn(service_type_=0, rec_log_ts={val:1742176609930802001, v:0}) [2025-03-17 09:56:57.673534] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=12] get rec log scn(service_type_=1, rec_log_ts={val:1742139624749258001, v:0}) [2025-03-17 09:56:57.673542] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=2, rec_log_ts={val:1742138690841156001, v:0}) [2025-03-17 09:56:57.673557] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:57.673579] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=19] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:57.673596] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742067167143792000, v:0}, checkpoint_scn_in_ls_meta={val:1742067167143792000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:56:57.673617] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=13][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4831543296}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:0, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4831543296}, lsn={lsn:4831543296}) [2025-03-17 09:56:57.673642] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=24] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4831543296}, ls->get_ls_id()={id:1}) [2025-03-17 09:56:57.673656] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=12] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2025-03-17 09:56:57.673759] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=25] ====== checkpoint timer task ====== [2025-03-17 09:56:57.673792] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1742176617153351004, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:872700639}, palf_committed_end_scn_:{val:1742176617654369000, v:0}, last_check_scn_:{val:1742176617654369000, v:0}, max_applied_cb_scn_:{val:1742176617153351004, v:0}}) [2025-03-17 09:56:57.673810] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=15] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595957515623, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79ae32fa70, log_storage_:{log_tail:{lsn:872700639}, readable_log_tail:{lsn:872700639}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:872361984}, min_scn:{val:1742176383843265001, v:0}, curr_block_id:13, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1/log", dir_fd:275, min_block_id:0, max_block_id:14, curr_writable_block_id:13}, logical_block_size_:67104768, curr_block_writable_size_:66766113, block_header_serialize_buf_:0x7f79ae330738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:56:57.673914] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=21] gc stale ls task succ [2025-03-17 09:56:57.674498] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=45] succ to get rec scn(*this={this:0x7f79fc004030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:2, v:0}, max_applied_start_working_ts_:{val:2, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595954890076, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:713}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.674520] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=21] tx ctx memtable get rec scn(this={ObITable:{this:0x7f79ae3a8080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f79ae3a8080, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.674658] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=11] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742148082680599002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1742148082680599002, v:0}}, }, *this={this:0x7f79ae2db150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176614150155004, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176627787617000, latest_log_ts:{val:1742176611145419001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.674713] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=52] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7f79ae2da1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7f79ae2e5010, memtable_head:19, memtable_tail:20, t3m:0x7f79f86ea030, tables:[null, null, null, 0x7f79fac1a7b0, null, null, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1742148004181715, tx_data_table:0x7f79ae2e6d10, ls_tablet_svr:0x7f79ae2da190}, min_rec_scn={val:1742148008655744000, v:0}, ls_id_={id:1}) [2025-03-17 09:56:57.674733] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=17] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1, v:0}) [2025-03-17 09:56:57.674743] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1742176611145419001, v:0}) [2025-03-17 09:56:57.674750] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.674759] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=6] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:57.674773] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:57.674780] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=6] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742148008655744000, v:0}, checkpoint_scn_in_ls_meta={val:1742148008655744000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:56:57.674794] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:805257216}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:805257216}, lsn={lsn:805257216}) [2025-03-17 09:56:57.674809] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=14] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:805257216}, ls->get_ls_id()={id:1}) [2025-03-17 09:56:57.674824] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=7] get_max_applied_scn(scn={val:1742176617153351004, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:4162412784}, palf_committed_end_scn_:{val:1742176617654369000, v:0}, last_check_scn_:{val:1742176617654369000, v:0}, max_applied_cb_scn_:{val:1742176617153351004, v:0}}) [2025-03-17 09:56:57.674836] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=10] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1741595966225063000, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595979082691, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741595966225063000, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79c4aa1a70, log_storage_:{log_tail:{lsn:4162412784}, readable_log_tail:{lsn:4162412784}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4160495616}, min_scn:{val:1742176364319009003, v:0}, curr_block_id:62, palf_id:1001}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1001/log", dir_fd:274, min_block_id:0, max_block_id:63, curr_writable_block_id:62}, logical_block_size_:67104768, curr_block_writable_size_:65187600, block_header_serialize_buf_:0x7f79c4aa2738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1001, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:56:57.675500] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=40] succ to get rec scn(*this={this:0x7f79fc0b6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1741595966225063001, v:0}, max_applied_start_working_ts_:{val:1741595966225063001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595976091006, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:246}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.675532] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x7f79ae3a9350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f79ae3a9350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:57.675582] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=15] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742148095278530005, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200048}, rec_scn:{val:1742148095278530005, v:0}}, }, *this={this:0x7f79e6d15150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148081676232000, v:0}, clog_base_lsn:{lsn:4093390848}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176614150155004, v:0}, tablet_change_checkpoint_scn:{val:1742008506458625001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.675641] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=58] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7f79e6d141b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7f79e6d1f010, memtable_head:12, memtable_tail:13, t3m:0x7f79f86ea030, tables:[null, null, null, null, null, null, null, null, null, null, null, null, 0x7f79fac18080, null, null, null]}, is_freezing:false, ls_id:{id:1001}, mini_merge_recycle_commit_versions_ts:1742148004203272, tx_data_table:0x7f79e6d20d10, ls_tablet_svr:0x7f79e6d14190}, min_rec_scn={val:1742148081676232000, v:0}, ls_id_={id:1001}) [2025-03-17 09:56:57.675661] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=18] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1741595966225063000, v:0}) [2025-03-17 09:56:57.675673] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2025-03-17 09:56:57.675687] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=12] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2025-03-17 09:56:57.675699] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742148081676232000, v:0}, checkpoint_scn_in_ls_meta={val:1742148081676232000, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2025-03-17 09:56:57.675710] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4093390848}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1741595966225063000, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4093390848}, lsn={lsn:4093390848}) [2025-03-17 09:56:57.675726] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4093390848}, ls->get_ls_id()={id:1001}) [2025-03-17 09:56:57.675735] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=8] succeed to update_clog_checkpoint(ret=0, ls_cnt=2) [2025-03-17 09:56:57.679161] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=44][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.685688] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463503664}], scn_array=[cnt:1, {val:1742176617661790000, v:0}], curr_size=514, accum_size=4165, time_cost=23605) [2025-03-17 09:56:57.696892] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=31] current monitor number(seq_=-1) [2025-03-17 09:56:57.708253] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=9] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:57.709675] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=47] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.203409972190856934e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=601, Throttle Trigger(MB)=361, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.350672783926843863e+02) [2025-03-17 09:56:57.710633] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.710667] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=33][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:57.710684] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=16][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:57.710692] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c4b00, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F7-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176617710136, enqueue_used:2, dequeue_used:25, submit_used:1, return_used:443, callback_enqueue_used:2, callback_dequeue_used:20, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.710734] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c4b00, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F7-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176617710136, enqueue_used:2, dequeue_used:25, submit_used:1, return_used:443, callback_enqueue_used:2, callback_dequeue_used:20, callback_finish_used:-1, end_used:1742176617710726}}, part_id:-1}) [2025-03-17 09:56:57.710971] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=21][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:57.710985] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=14][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:57.710992] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a2281515a, src_buf_size=1611) [2025-03-17 09:56:57.711001] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7ad0, ref_cnt_:1, raw_buf_:0x7f7a22814150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F7-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176617710766, enqueue_used:1, dequeue_used:13, submit_used:1, return_used:174, callback_enqueue_used:0, callback_dequeue_used:14, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:57.711026] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=19][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7ad0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F7-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176617710766, enqueue_used:1, dequeue_used:13, submit_used:1, return_used:174, callback_enqueue_used:0, callback_dequeue_used:14, callback_finish_used:-1, end_used:1742176617711021}}, part_id:-1}) [2025-03-17 09:56:57.711172] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222519955, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2352a0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:57.711215] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634298} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:57.711238] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=21] end stmt(ret=0, tx_id=9634298, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2352a0, tx_id:{txid:9634298}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519955, assoc_session_id:3222519955, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176617708440, active_ts:1742176617708440, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:1742176647708432, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461824123118, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617707413}], exec_info_reap_ts:1742176617707412, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617707409, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222519955, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2352a0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:57.711345] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=1][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634298}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176647708432, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617709505}{self_ls_id:{id:1}, session_id:3222519955, associated_session_id:3222519955, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634298}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461824123118, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617709505, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634298} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.711403] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=48] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634298}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176647708432, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176617709505}{self_ls_id:{id:1}, session_id:3222519955, associated_session_id:3222519955, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634298}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409461824123118, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176617709505, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634298} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:57.711460] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=52] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634298}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176617711342, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:57.711476] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2352a0, tx_id:{txid:9634298}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519955, assoc_session_id:3222519955, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176617708440, active_ts:1742176617708440, commit_ts:-1, finish_ts:1742176617710568, timeout_us:29999992, lock_timeout_us:-1, expire_ts:1742176647708432, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409461824123118, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176617707413}], exec_info_reap_ts:1742176617707412, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176617707409, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:57.711554] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F7-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:57.735042] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=41] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:56:57.745230] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=4752) [2025-03-17 09:56:57.778323] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=20] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:57.779209] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.779578] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=53][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463504178}], scn_array=[cnt:1, {val:1742176617744568000, v:0}], curr_size=587, accum_size=4752, time_cost=34336) [2025-03-17 09:56:57.779664] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162412784}], scn_array=[cnt:1, {val:1742176617654369001, v:0}], curr_size=122, accum_size=732, time_cost=21686) [2025-03-17 09:56:57.779758] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=7] [APPLY STAT CB APPEND COST TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=14, this=0x7f79e6df6280, extra_info="") [2025-03-17 09:56:57.779785] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=26] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=9904, this=0x7f79e6df64b0, extra_info="") [2025-03-17 09:56:57.779797] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=12] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=0, this=0x7f79e6df66e0, extra_info="") [2025-03-17 09:56:57.779808] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=10] [APPLY STAT CB EXECUTE TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=-541, this=0x7f79e6df6910, extra_info="") [2025-03-17 09:56:57.779816] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=8] [APPLY STAT CB TOTAL TIME](cur_stat_count=22, stat_interval=5000000, avg (count/cost)=9377, this=0x7f79e6df6b40, extra_info="") [2025-03-17 09:56:57.781703] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12224, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:56:57.781729] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872700639}], scn_array=[cnt:1, {val:1742176617654369001, v:0}], curr_size=122, accum_size=610, time_cost=23747) [2025-03-17 09:56:57.781770] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=27, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:56:57.781799] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=5, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:56:57.781815] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12332, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:56:57.781835] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:56:57.785988] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40309312716, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:57.793009] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1003, min_version={val:1742176617593499000, v:0}, base_version={val:1742176617394213000, v:0}, server_count=1, svr_array=[{addr:"192.168.1.62:2882", version:{val:1742176617593499000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1742176617792995, is_skipped:false}], skip_server_count=0, skip_servers=[]) [2025-03-17 09:56:57.795429] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=58055, gc_interval=10000000) [2025-03-17 09:56:57.795479] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=33] self is leader, skip it(ls->get_ls_id()={id:1}) [2025-03-17 09:56:57.795492] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=10] self is leader, skip it(ls->get_ls_id()={id:1001}) [2025-03-17 09:56:57.795504] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] gc_check_member_list_ cost time(ret=0, time_us=42) [2025-03-17 09:56:57.795513] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=0) [2025-03-17 09:56:57.796517] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2025-03-17 09:56:57.797274] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=24] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}) [2025-03-17 09:56:57.797303] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=25] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}]) [2025-03-17 09:56:57.797313] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1001}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}, {ls_id_:{id:1001}, ls_status_:1, gc_reason_:0}]) [2025-03-17 09:56:57.797319] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] execute_gc cost time(ret=0, time_us=17) [2025-03-17 09:56:57.797324] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=4] ObSafeDestroyHandler start process [2025-03-17 09:56:57.797330] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:133) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ObSafeDestroyTaskQueue::loop begin(queue_.size()=0) [2025-03-17 09:56:57.797345] INFO [SERVER] loop (ob_safe_destroy_handler.cpp:140) [3539231][T1004_GCCollect][T1004][Y0-0000000000000000-0-0] [lt=5] ObSafeDestroyTaskQueue::loop finish(ret=0, queue_.size()=0) [2025-03-17 09:56:57.805326] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2747) [3539406][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=34] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", [SlidingWindow]={palf_id:1, self:"192.168.1.62:2882", lsn_allocator:{max_log_id:8109585, max_lsn:4859682575, max_scn:{val:1742176617793323000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:4859681987}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:4817739535}}, last_submit_lsn:{lsn:4859681987}, last_submit_end_lsn:{lsn:4859682575}, last_submit_log_id:8109585, last_submit_log_pid:1, max_flushed_lsn:{lsn:4859681400}, max_flushed_end_lsn:{lsn:4859681987}, max_flushed_log_pid:1, committed_end_lsn:{lsn:4859681987}, last_slide_log_id:8109584, last_slide_scn:{val:1742176617730762000, v:0}, last_slide_lsn:{lsn:4859681400}, last_slide_end_lsn:{lsn:4859681987}, last_slide_log_pid:1, last_slide_log_accum_checksum:2645468327, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f79aaff0530}, [StateMgr]={this:0x7f79aaff4830, self:"192.168.1.62:2882", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"192.168.1.62:2882", leader_epoch:0, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:8109585, is_changing_config_with_arb:false, reconfirm_start_time_us:1741595945778042, palf_role_change_cb:0x7f79aaffb2f0, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"192.168.1.62:2882", alive_paxos_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, alive_paxos_replica_num:1, log_ms_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, reconfig_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:1, config_seq:2}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f79aaff1c70}, [ModeMgr]={palf_id:1, self:"192.168.1.62:2882", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:1741697795543680001, v:0}, min_block_id:12, min_block_min_scn:{val:1741689673128951001, v:0}, base_lsn_for_block_gc:{lsn:4697333760}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:2, base_lsn:{lsn:4697333760}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:65536}, readable_log_tail:{lsn:65536}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4898648064}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003/1/meta", dir_fd:262, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67039232, block_header_serialize_buf_:0x7f79aaff6a38, flashback_version:0}, log_storage:{log_tail:{lsn:4859681987}, readable_log_tail:{lsn:4859681987}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4831543296}, min_scn:{val:1742173162720054001, v:0}, curr_block_id:72, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003/1/log", dir_fd:268, min_block_id:12, max_block_id:73, curr_writable_block_id:72}, logical_block_size_:67104768, curr_block_writable_size_:38966077, block_header_serialize_buf_:0x7f79aaff8738, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1741595945777450, this:0x7f79aaff4f70}, [Reconfirm]={palf_id:1, self:"192.168.1.62:2882", state:"FINISHED", new_proposal_id:1, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:1, majority_max_log_server:"192.168.1.62:2882", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:0}, saved_end_lsn:{lsn:0}, last_submit_prepare_req_time_us:1741595945778239, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f79aaff49b0}) [2025-03-17 09:56:57.805551] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2751) [3539406][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=220] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176617734898, last_flushed_end_lsn:{lsn:4859681987}}]) [2025-03-17 09:56:57.805568] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4325) [3539406][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=15] current log_task status(palf_id=1, self="192.168.1.62:2882", log_id=8109585, log_task={header:{begin_lsn:{lsn:4859681987}, end_lsn:{lsn:4859682575}, log_id:8109585, min_scn:{val:1742176617793323000, v:0}, max_scn:{val:1742176617793323000, v:0}, data_len:532, proposal_id:1, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:4859681987}, data_checksum:4037275769, accum_checksum:338448659, is_padding_log:false, is_raw_write:false}, state_map:{val:30}, ref_cnt:0, gen_ts:1742176617794218, freeze_ts:1742176617794222, submit_ts:1742176617794232, flushed_ts:-1, gen_to_freeze cost time:4, gen_to_submit cost time:14, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176617734898, last_flushed_end_lsn:{lsn:4859681987}}]) [2025-03-17 09:56:57.830013] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=39] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1742176617654369000, v:0}, total_part_count:2, valid_inner_part_count:1, valid_user_part_count:1}, server_version_delta=175635, in_cluster_service=true, cluster_version={val:1742176617153351003, v:0}, min_cluster_version={val:1742176617153351003, v:0}, max_cluster_version={val:1742176617153351003, v:0}, get_cluster_version_err=0, cluster_version_delta=676653, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=5759541, succ_cluster_heartbeat_count=1, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176143516287002, v:0}, local_cluster_delta=474313717, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:56:57.830879] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859681987}], scn_array=[cnt:1, {val:1742176617793323000, v:0}], curr_size=588, accum_size=3649, time_cost=36610) [2025-03-17 09:56:57.840764] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=22] ====== traversal_flush timer task ====== [2025-03-17 09:56:57.841105] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539208][T1004_TimerWK2][T1004][YB420A0B093E-00062FB958B5C0E9-0-0] [lt=41] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1742148082680599002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1742148082680599003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1742148082680599004, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1742148082680599005, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1742148082680599006, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1742148082680599008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1742148082680599009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1742148082680599010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1742148082680599012, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1742148082680599013, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1742148082680599014, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1742148082680599015, v:0}}, {tablet_id:{id:101109}, rec_scn:{val:1742148082680599017, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1742148082680599018, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1742148082680599019, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1742148083181692001, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1742148083181692002, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1742148083181692003, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1742148083181692004, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1742148083181692005, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1742148083181692007, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1742148083181692008, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1742148083181692009, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1742148083181692010, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1742148083181692011, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1742148083181692012, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1742148083181692013, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1742148083181692015, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1742148083181692016, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1742148083181692017, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1742148083181692018, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1742148083181692019, v:0}}, }, max_consequent_callbacked_scn={val:1742176617654369001, v:0}, *this={this:0x7f79ae2db150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176614150155004, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176627787617000, latest_log_ts:{val:1742176611145419001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.841458] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539208][T1004_TimerWK2][T1004][YB420A0B093E-00062FB958B5C0E9-0-0] [lt=232] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200048}, rec_scn:{val:1742148095278530005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1742148095278530006, v:0}}, {tablet_id:{id:1152921504606847070}, rec_scn:{val:1742148095278530007, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1742148095278530008, v:0}}, {tablet_id:{id:1152921504606847061}, rec_scn:{val:1742148095697591001, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1742148095697591003, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1742148095697591004, v:0}}, {tablet_id:{id:1152921504606847037}, rec_scn:{val:1742148095697591005, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1742148095697591006, v:0}}, {tablet_id:{id:200052}, rec_scn:{val:1742148095697591007, v:0}}, {tablet_id:{id:200063}, rec_scn:{val:1742148095697591008, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1742148095697591009, v:0}}, {tablet_id:{id:1152921504606847065}, rec_scn:{val:1742148095697591011, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1742148095697591012, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1742148095697591013, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1742148095697591014, v:0}}, {tablet_id:{id:200086}, rec_scn:{val:1742148095697591015, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1742148095697591017, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1742148095697591018, v:0}}, {tablet_id:{id:1152921504606847046}, rec_scn:{val:1742148095697591020, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1742148095697591021, v:0}}, {tablet_id:{id:200059}, rec_scn:{val:1742148096198618001, v:0}}, {tablet_id:{id:200145}, rec_scn:{val:1742148096198618003, v:0}}, {tablet_id:{id:200034}, rec_scn:{val:1742148096198618005, v:0}}, {tablet_id:{id:200122}, rec_scn:{val:1742148096198618006, v:0}}, {tablet_id:{id:1152921504606847049}, rec_scn:{val:1742148096198618007, v:0}}, {tablet_id:{id:1152921504606847057}, rec_scn:{val:1742148096198618008, v:0}}, {tablet_id:{id:1152921504606847068}, rec_scn:{val:1742148096698960001, v:0}}, {tablet_id:{id:1152921504606847105}, rec_scn:{val:1742148096698960003, v:0}}, {tablet_id:{id:200025}, rec_scn:{val:1742148096698960004, v:0}}, {tablet_id:{id:1152921504606847101}, rec_scn:{val:1742148096698960005, v:0}}, {tablet_id:{id:1152921504606847036}, rec_scn:{val:1742148096698960006, v:0}}, }, max_consequent_callbacked_scn={val:1742176617654369001, v:0}, *this={this:0x7f79e6d15150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148081676232000, v:0}, clog_base_lsn:{lsn:4093390848}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176614150155004, v:0}, tablet_change_checkpoint_scn:{val:1742008506458625001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:57.841683] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [3539208][T1004_TimerWK2][T1004][YB420A0B093E-00062FB958B5C0E9-0-0] [lt=227] succeed to traversal_flush(ret=0, ls_cnt=2) [2025-03-17 09:56:57.851454] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=7] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=9) [2025-03-17 09:56:57.857222] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [3539234][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=31] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:56:57.857990] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae329470, extra_info="") [2025-03-17 09:56:57.858031] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:56:57.858175] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=20] QSched: |cfg: root:1,x,0 net_in:1,1248M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:312K/75:14210 disk:312K/75:14210 t1m2:136K/33:13278 t1m2g0:136K/33:13278 t1004m2:84K/20:13138 t1004m2g0:84K/20:13138 t1003m2:92K/22:16584 t1003m2g0:92K/22:16584 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:56:57.876607] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162412906}], scn_array=[cnt:1, {val:1742176617654369002, v:0}], curr_size=122, accum_size=854, time_cost=18537) [2025-03-17 09:56:57.879259] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.882694] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=53] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=16895, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:56:57.882727] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=4874) [2025-03-17 09:56:57.882744] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=25, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:56:57.882764] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=59] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=4874) [2025-03-17 09:56:57.882814] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=35] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=8, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:56:57.882827] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=17014, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:56:57.882812] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=28] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107779, offset=57285897}, end_cursor:ObLogCursor{file_id=5, log_id=1107780, offset=57286203}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b176dc050, len:759, log_data_len:306, seq:1107779, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:57.882843] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=41, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:56:57.882968] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:56:57.883002] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=21] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176617654369001, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176617654369001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:57.885197] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=12][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:57.885319] INFO [SHARE] update_tenant_recovery_status_in_trans (ob_tenant_info_proxy.cpp:491) [3539176][T1004_TeRec][T1004][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=26] tenant scn gap info(IS_MAX_GAP_REACHED=false, REAL_GAP=0, MAX_GAP=900000000000, new_sync_scn={val:1742176616652787001, v:0}, new_replayable_scn={val:1742176616652787001, v:0}, new_readable_scn={val:1742176616652787001, v:0}, old_tenant_info={tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176615651502000, replayable_scn:1742176615651502000, readable_scn:1742176615150772004, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}) [2025-03-17 09:56:57.885589] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9258, avg_group_log_size=578) [2025-03-17 09:56:57.918949] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=36] Slow write(duration=25674, write_len=4096) [2025-03-17 09:56:57.918982] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=32] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107780, offset=57286203}, end_cursor:ObLogCursor{file_id=5, log_id=1107781, offset=57286500}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b1772c050, len:453, log_data_len:297, seq:1107780, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:57.919124] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:56:57.919173] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED7319-0-0] [lt=37] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176617654369001, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176617654369001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:57.927504] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:56:57.927534] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=29][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:56:57.937758] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=7] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312799531030, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:56:57.937802] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=40][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:56:57.937816] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=13] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7aeb1ff690) [2025-03-17 09:56:57.937824] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=7] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312799531030, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:56:57.945316] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:56:57.945357] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:56:57.958086] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:56:57.960739] INFO [SERVER] refresh_avail_memory (ob_table_load_assigned_memory_manager.cpp:117) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=10] ObTableLoadAssignedMemoryManager::refresh_avail_memory(MTL_ID()=1, avail_memory=322122547, chunk_count_=0, avail_sort_memory_=322122547, avail_memory_=322122547) [2025-03-17 09:56:57.963018] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859683707}], scn_array=[cnt:1, {val:1742176617920271000, v:0}], curr_size=749, accum_size=5530, time_cost=42118) [2025-03-17 09:56:57.966905] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=5] Garbage Collector is running(seq_=85972, gc_interval=10000000) [2025-03-17 09:56:57.966957] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=35] self is leader, skip it(ls->get_ls_id()={id:1}) [2025-03-17 09:56:57.966973] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=9] gc_check_member_list_ cost time(ret=0, time_us=33) [2025-03-17 09:56:57.966984] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=1) [2025-03-17 09:56:57.966993] INFO [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1755) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=6] gc_candidates push_back success(ret=0, candidate={ls_id_:{id:1}, ls_status_:1, gc_reason_:0}) [2025-03-17 09:56:57.967005] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1777) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=9] ls status is normal, skip(id={id:1}, gc_candidates=[{ls_id_:{id:1}, ls_status_:1, gc_reason_:0}]) [2025-03-17 09:56:57.967013] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=8] execute_gc cost time(ret=0, time_us=9) [2025-03-17 09:56:57.967028] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [117803][T1_GCCollector][T1][Y0-0000000000000000-0-0] [lt=13] ObSafeDestroyHandler start process [2025-03-17 09:56:57.973514] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1089) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=25] replay status is not follower(min_unreplayed_lsn={lsn:0}, base_lsn={lsn:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595943665555, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79aaff7a70, log_storage_:{log_tail:{lsn:4859684456}, readable_log_tail:{lsn:4859684456}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4831543296}, min_scn:{val:1742173162720054001, v:0}, curr_block_id:72, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003/1/log", dir_fd:268, min_block_id:12, max_block_id:73, curr_writable_block_id:72}, logical_block_size_:67104768, curr_block_writable_size_:38963608, block_header_serialize_buf_:0x7f79aaff8738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:56:57.973603] INFO [CLOG] operator() (ob_log_replay_service.cpp:1679) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=88] get_replay_process success(id={id:1}, submitted_log_size=0, unsubmitted_log_size=0, replayed_log_size=0, unreplayed_log_size=0) [2025-03-17 09:56:57.973627] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:182) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=9] dump tenant replay process(tenant_id=1003, unsubmitted_log_size(MB)=0, unreplayed_log_size(MB)=0, estimate_time(second)=0, submitted_log_size(MB)=0, replayed_log_size(MB)=0, last_submitted_log_size(MB)=0, last_replayed_log_size(MB)=0, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2025-03-17 09:56:57.979075] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463504887}], scn_array=[cnt:1, {val:1742176617944805000, v:0}], curr_size=586, accum_size=586, time_cost=33701) [2025-03-17 09:56:57.979167] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872700883}], scn_array=[cnt:1, {val:1742176617654369003, v:0}], curr_size=122, accum_size=854, time_cost=21089) [2025-03-17 09:56:57.979321] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=55][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:57.979541] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=55][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162413028}], scn_array=[cnt:1, {val:1742176617654369003, v:0}], curr_size=122, accum_size=976, time_cost=21366) [2025-03-17 09:56:57.982966] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859684456}], scn_array=[cnt:1, {val:1742176617930922000, v:0}], curr_size=588, accum_size=6118, time_cost=19862) [2025-03-17 09:56:57.983947] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA583-0-0] [lt=1][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:57.986264] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=9] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40305118412, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:57.994656] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=37] [PALF STAT APPEND COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:56:58.011097] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=7] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:58.011183] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:58.028806] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859685044}], scn_array=[cnt:1, {val:1742176617993858000, v:0}], curr_size=586, accum_size=6704, time_cost=34131) [2025-03-17 09:56:58.030179] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=56] [WRS] update tenant weak read server version(tenant_id=1004, server_version={version:{val:1742176617654369003, v:0}, total_part_count:2, valid_inner_part_count:1, valid_user_part_count:1, epoch_tstamp:1742176618030138}, version_delta=375808) [2025-03-17 09:56:58.051584] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=29] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=8) [2025-03-17 09:56:58.058170] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=44] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:56:58.062749] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=15] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=1, size_used=65536, mem_used=124784640) [2025-03-17 09:56:58.062806] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=24] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:58.079384] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.108568] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=8124) [2025-03-17 09:56:58.118314] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=21] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:56:58.120574] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:56:58.120622] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:10, fail_bw:40863, [delay/us]:prepare:1, schedule:18, submit:0, rt:2496, total:2516, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:58.120640] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4468, iops:11, bw:49036, [delay/us]:prepare:5, schedule:22, submit:20, rt:19549, total:19597) [2025-03-17 09:56:58.120651] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=2, iops=2, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:58.120686] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=31] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:10, avg_bw:40870, [delay/us]: prepare:1, schedule:18, submit:0, device:2496, total:2516) [2025-03-17 09:56:58.120699] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4468, avg_iops:11, avg_bw:49045, [delay/us]: prepare:5, schedule:22, submit:20, device:19549, total:19597) [2025-03-17 09:56:58.120712] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:56:58.120740] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32699, [delay/us]:prepare:1, schedule:29, submit:1, rt:4718, total:4750, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:58.120761] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4681, iops:14, bw:65397, [delay/us]:prepare:6, schedule:22, submit:1163, rt:24285, total:25477) [2025-03-17 09:56:58.120770] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:58.120789] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32701, [delay/us]: prepare:1, schedule:29, submit:1, device:4718, total:4750) [2025-03-17 09:56:58.120805] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4681, avg_iops:14, avg_bw:65403, [delay/us]: prepare:6, schedule:22, submit:1163, device:24285, total:25477) [2025-03-17 09:56:58.120817] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:56:58.120836] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8175, [delay/us]:prepare:5, schedule:25, submit:1, rt:14545, total:14577) [2025-03-17 09:56:58.120856] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4300, iops:20, bw:85845, [delay/us]:prepare:5, schedule:19, submit:1220, rt:12516, total:13762) [2025-03-17 09:56:58.120865] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:58.120883] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8176, [delay/us]: prepare:5, schedule:25, submit:1, device:14545, total:14577) [2025-03-17 09:56:58.120897] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4301, avg_iops:20, avg_bw:85853, [delay/us]: prepare:5, schedule:19, submit:1220, device:12516, total:13762) [2025-03-17 09:56:58.120915] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:56:58.120939] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4088, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:294, total:298) [2025-03-17 09:56:58.120965] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:58.120983] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4088, [delay/us]: prepare:0, schedule:0, submit:600000000, device:294, total:298) [2025-03-17 09:56:58.120999] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:56:58.121010] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:58.121019] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121026] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121032] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121038] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121045] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121051] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121057] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121064] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:58.121071] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121077] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121083] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121090] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:58.121099] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SERVER] net_in=175kB/s, net_out=73kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:56:58.128503] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=9] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=24956928) [2025-03-17 09:56:58.128559] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=12] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:58.150092] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=52][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859685630}], scn_array=[cnt:1, {val:1742176617993858001, v:0}], curr_size=122, accum_size=6826, time_cost=41504) [2025-03-17 09:56:58.153059] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:449) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=8] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:58.153084] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=25] try reload schema success(schema_version=1741661297536408, refresh_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_ret=0) [2025-03-17 09:56:58.153076] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:263) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A0-0-0] [lt=9] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:58.153098] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:288) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A0-0-0] [lt=20] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:56:58.154859] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=9] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:56:58.155530] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570E-0-0] [lt=29] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11027, add_timestamp:1742176618155519}) [2025-03-17 09:56:58.155545] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3185-0-0] [lt=19] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11027, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:58.155657] INFO [SERVER] fill_tenant_ids_ (ob_all_virtual_tx_stat.cpp:141) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570E-0-0] [lt=23] succeed to get tenant ids(all_tenants_=[1, 1003, 1004]) [2025-03-17 09:56:58.155747] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3185-0-0] [lt=20] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11027, locations=[{table_id:11027, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:58.155774] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3185-0-0] [lt=27] success to process renew task(task={tenant_id:1, table_id:11027, add_timestamp:1742176618155519}, locations=[{table_id:11027, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}, {table_id:11027, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176618155734, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:58.170655] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:559) [117778][T1_HBService][T1][YB420A0B093E-00062FB7F27BCA19-0-0] [lt=14] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2025-03-17 09:56:58.174376] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=32] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:56:58.179189] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463505595}], scn_array=[cnt:1, {val:1742176618144409000, v:0}], curr_size=587, accum_size=1295, time_cost=33933) [2025-03-17 09:56:58.179297] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=18, avg_log_submit_to_flush_time=22305, avg_log_submit_to_slide_time=22322) [2025-03-17 09:56:58.179343] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=4752, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:58.179355] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859685752}], scn_array=[cnt:1, {val:1742176618131661000, v:0}], curr_size=588, accum_size=7414, time_cost=29158) [2025-03-17 09:56:58.179500] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=55][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.181632] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=21] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:56:58.182794] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701127}], scn_array=[cnt:1, {val:1742176618154213000, v:0}], curr_size=122, accum_size=1098, time_cost=24578) [2025-03-17 09:56:58.182853] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15543, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:56:58.182872] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162413272}], scn_array=[cnt:1, {val:1742176618154213000, v:0}], curr_size=122, accum_size=1220, time_cost=24599) [2025-03-17 09:56:58.182894] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:56:58.182943] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=44] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=23, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:56:58.182997] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:56:58.183045] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:56:58.183055] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15641, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:56:58.183066] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=35, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:56:58.183185] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:775) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=13] success to update min reserved snapshot(reserved_snapshot=1742174807617607000, duration=1800, snapshot_gc_ts=1742176607617607000) [2025-03-17 09:56:58.183548] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=28] PNIO [ratelimit] time: 1742176618183546, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007612, add_bytes: 0 [2025-03-17 09:56:58.186585] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=37] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40305118412, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:58.190825] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=13] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:56:58.191558] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=23] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:56:58.191616] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=18] inner load succ(latest_freeze_info={frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615679197000, v:0}, frozen_statuses_.count():8, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}]}) [2025-03-17 09:56:58.193326] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463506182}], scn_array=[cnt:1, {val:1742176618144409001, v:0}], curr_size=112, accum_size=1407, time_cost=10071) [2025-03-17 09:56:58.193451] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [117806][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=22] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:56:58.193604] INFO [STORAGE] update_min_reserved_snapshot_for_leader (ob_ls_reserved_snapshot_mgr.cpp:191) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=21] submit reserved snapshot log success(ls_id={id:1}, new_snapshot_version=1742174807617607000) [2025-03-17 09:56:58.193629] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:802) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=21] update reserved snapshot finished(cost_ts=10445, reserved_snapshot=1742174807617607000) [2025-03-17 09:56:58.193989] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:671) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=28] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=0, clean_start_pos=5600000, clean_num=200000) [2025-03-17 09:56:58.194324] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC5-0-0] [lt=25] [RS_MGR] force set rs list(master_rs="192.168.1.62:2882") [2025-03-17 09:56:58.194344] INFO [SERVER] handle_heartbeat (ob_service.cpp:3753) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC5-0-0] [lt=16] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"192.168.1.62:2882", server_id:4, rs_addr:"192.168.1.62:2882", rs_server_status:1, epoch_id:4}, hb_response={zone:"zone1", server:"192.168.1.62:2882", sql_port:2881, build_version:"4.3.5.0_100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5(Dec 31 2024 17:35:01)", start_service_time:1741316885024955, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=22) [2025-03-17 09:56:58.212717] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=31] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:56:58.212767] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=39] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=8256) [2025-03-17 09:56:58.228266] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117882][T1_L5_G0][T1][YB420A0B0940-00063023B53C2B4C-0-0] [lt=22] get number(ret=0, service_type_=0, range=1, base_id=1742176618227844000, start_id=1742176618227844000, end_id=1742176618227844001) [2025-03-17 09:56:58.230133] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859686340}], scn_array=[cnt:1, {val:1742176618193719000, v:0}], curr_size=588, accum_size=8002, time_cost=35679) [2025-03-17 09:56:58.242926] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=13][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-07 11:08:01.882057|total=859736.36s) [2025-03-17 09:56:58.242973] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=46][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-08 02:00:05.527466|total=806212.72s) [2025-03-17 09:56:58.242982] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=8][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-09 02:00:02.826568|total=719815.42s) [2025-03-17 09:56:58.242988] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-10 02:00:03.664820|total=633414.58s) [2025-03-17 09:56:58.242994] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-11 02:00:06.154807|total=547012.09s) [2025-03-17 09:56:58.242999] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=5][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-11 02:00:02.2305|total=547016.24s) [2025-03-17 09:56:58.243005] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-12 02:00:05.732930|total=460612.51s) [2025-03-17 09:56:58.243011] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-11 02:00:05.267002|total=547012.98s) [2025-03-17 09:56:58.243016] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=5][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-12 02:00:02.599724|total=460615.64s) [2025-03-17 09:56:58.243022] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-13 02:00:02.834671|total=374215.41s) [2025-03-17 09:56:58.243028] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=6][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-14 02:00:05.539277|total=287812.70s) [2025-03-17 09:56:58.243038] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=10][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-15 02:00:01.306225|total=201416.94s) [2025-03-17 09:56:58.243049] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=10][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-16 02:00:05.166204|total=115013.08s) [2025-03-17 09:56:58.243057] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=8][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-17 02:00:04.960897|total=28613.28s) [2025-03-17 09:56:58.243066] WDIAG [STORAGE] ~ObCompactionTimeGuard (ob_compaction_time_guard.cpp:28) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB99FB767B2-0-0] [lt=8][errcode=0] cost too much time(this=|threshold=60.00s|start at 2025-03-17 02:00:04.960897|total=28613.28s) [2025-03-17 09:56:58.246389] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:58.246427] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:58.246720] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176616652787001, replayable_scn:1742176616652787001, readable_scn:1742176616652787001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176617884965000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:58.246744] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176616652787001, replayable_scn:1742176616652787001, readable_scn:1742176616652787001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176617884965000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:58.246756] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176616652787001, replayable_scn:1742176616652787001, readable_scn:1742176616652787001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176617884965000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:58.247586] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFA2-0-0] [lt=5][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:58.247623] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:56:58.247629] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=5][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:56:58.250952] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:677) [117487][pnio1][T0][YB420A0B0941-00063023C844D277-0-0] [lt=43][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:1114466538, pcode_:1861, hlen_:192, priority_:5, flags_:6151, tenant_id_:1001, priv_tenant_id_:0, session_id_:0, trace_id_:12378264242497, timeout_:200000, timestamp:1742176618250856, dst_cluster_id:-1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:65974784, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1741272511, seq_no_:1742176618250470, data_version_:0}, chid_:0, clen_:130, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1742176618250949, enqueue_timestamp_:0, request_arrival_time_:1742176618250949, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2025-03-17 09:56:58.251004] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:876) [117487][pnio1][T0][YB420A0B0941-00063023C844D277-0-0] [lt=44][errcode=-5150] deliver rpc request fail(&req=0x7f7970610098, ret=-5150) [2025-03-17 09:56:58.251726] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=12] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=7) [2025-03-17 09:56:58.253174] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [3539247][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=27] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f79c25fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:56:58.253573] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=28] ====== check clog disk timer task ====== [2025-03-17 09:56:58.253596] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=18] get_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:56:58.262849] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78622-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8580, ref_cnt_:1, raw_buf_:0x7f7a3f34e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78622-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a6033e080, time_log:{begin_ts:1742176618262391, enqueue_used:2, dequeue_used:40, submit_used:2, return_used:364, callback_enqueue_used:3, callback_dequeue_used:44, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:58.262918] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78622-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.262982] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78622-0-0] [lt=6][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:58.263463] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78622-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade96ac00, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519978, assoc_session_id:3222519978, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176618261994, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176618260920, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a393db118, id:3222519978, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade96ac00}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.279565] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.280406] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701249}], scn_array=[cnt:1, {val:1742176618154213001, v:0}], curr_size=122, accum_size=1220, time_cost=22131) [2025-03-17 09:56:58.280448] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:56:58.280486] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:56:58.280563] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:58.285840] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:723, free_obj_hold_cnt:78, allocator used:3082248, allocator total:3329280}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:233, free_obj_hold_cnt:724, allocator used:62770587, allocator total:62770587}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():130944, full allocator used:1600, full allocator total:196480}, tablets_mem=66296347, tablets_mem_limit=3092376440, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:88, allocator used:231424, allocator total:261120}, pending_cnt=0, wait_gc_count=0, tablet count=956) [2025-03-17 09:56:58.299266] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO [ratelimit] time: 1742176618299265, bytes: 2535005145443, bw: 0.148220 MB/s, add_ts: 1005140, add_bytes: 156219 [2025-03-17 09:56:58.308723] INFO [LIB] stat (utility.h:1210) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:56:58.320785] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A78-0-0] [lt=12] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:56:58.320816] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A78-0-0] [lt=30] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:56:58.330751] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859686928}], scn_array=[cnt:1, {val:1742176618193719001, v:0}], curr_size=122, accum_size=8124, time_cost=21954) [2025-03-17 09:56:58.342455] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=7] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:58.344352] INFO [STORAGE.TRANS] get_cluster_version (ob_tenant_weak_read_cluster_version_mgr.cpp:166) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [WEAK_READ_SERVER_VERSION_MGR] compute version(tenant_id=1, min_version={val:1742176618144409000, v:0}, base_version={val:1742176617944805000, v:0}, server_count=4, svr_array=[{addr:"192.168.1.62:2882", version:{val:1742176618144409000, v:0}, valid_part_count:1, total_part_count:1, generate_tstamp:1742176618344336, is_skipped:false}, {addr:"192.168.1.64:2882", version:{val:1742176613443737000, v:0}, valid_part_count:0, total_part_count:0, generate_tstamp:1742176618294708, is_skipped:true}, {addr:"192.168.1.65:2882", version:{val:1742176613380698000, v:0}, valid_part_count:0, total_part_count:0, generate_tstamp:1742176618308328, is_skipped:true}, {addr:"192.168.1.63:2882", version:{val:1742176613477774000, v:0}, valid_part_count:0, total_part_count:0, generate_tstamp:1742176618249705, is_skipped:true}], skip_server_count=3, skip_servers=["192.168.1.64:2882", "192.168.1.65:2882", "192.168.1.63:2882"]) [2025-03-17 09:56:58.345578] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=51] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=5875, avg_group_log_size=451) [2025-03-17 09:56:58.347780] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=19] gc stale ls task succ [2025-03-17 09:56:58.366968] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [117834][T1_DBMSSched][T1][YB420A0B093E-00062FB7F38CDC91-0-0] [lt=24] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:56:58.369861] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:89) [117818][T1_ObXAHbWorker][T1][Y0-0000000000000000-0-0] [lt=55] XA scheduler heartbeat task statistics(avg_time=0) [2025-03-17 09:56:58.378742] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859687050}], scn_array=[cnt:1, {val:1742176618331116000, v:0}], curr_size=588, accum_size=8712, time_cost=47081) [2025-03-17 09:56:58.378799] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=56] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8712) [2025-03-17 09:56:58.378832] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=25, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:56:58.378870] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=11, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:56:58.378900] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=30866, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:56:58.378883] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=32] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8148, ref_cnt_:1, raw_buf_:0x7f79740b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F8-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618343636, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:35210, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:58.378917] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=3181, avg_log_gen_to_submit_time=3195, avg_log_submit_to_flush_time=27653, avg_log_submit_to_slide_time=27671) [2025-03-17 09:56:58.378927] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=42][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.378942] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=41, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:56:58.378951] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=23][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.378950] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8712, hit_cnt=16, read_cnt=16, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:58.378971] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=19][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.378980] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8148, ref_cnt_:1, raw_buf_:0x7f79740b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F8-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618343636, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:35210, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.379020] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=29][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8148, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F8-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618343636, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:35210, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176618379012}}, part_id:-1}) [2025-03-17 09:56:58.378959] INFO [STORAGE] init (ob_index_block_builder.cpp:571) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=10] init sstable index builder(ret=0, data_desc={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:47, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:47, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=42 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=45 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=46 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=47 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=48 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=49 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=50 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=51 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=52 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=53 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=54 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=55 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=56 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=57 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=58 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=59 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=60 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:1, micro_block_size:16384}, index_store_desc_={desc:{static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:true, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}}, container_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}, leaf_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:2095976}, &object_cleaner_=0x7f7acca40520) [2025-03-17 09:56:58.379120] INFO [STORAGE] init (ob_index_block_builder.cpp:3831) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=154] succeed to init ObIndexBlockRebuilder(task_idx=-1, task_type=2, device_handle_=NULL, index_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:true, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}, container_store_desc={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741917602909677000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}) [2025-03-17 09:56:58.379649] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.382103] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.382133] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=30][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.382141] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a0e5bd15a, src_buf_size=1611) [2025-03-17 09:56:58.382161] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=19][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7c80, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F8-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618379054, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:3009, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.382197] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=25][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7c80, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F8-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618379054, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:3009, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176618382188}}, part_id:-1}) [2025-03-17 09:56:58.382230] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463506840}], scn_array=[cnt:1, {val:1742176618344925000, v:0}], curr_size=587, accum_size=2540, time_cost=36636) [2025-03-17 09:56:58.382343] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701371}], scn_array=[cnt:1, {val:1742176618154213002, v:0}], curr_size=122, accum_size=122, time_cost=24022) [2025-03-17 09:56:58.382343] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222520162, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf38b200}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:58.382366] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:219) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=64] Async write macro block(macro_id_={[ver=1,mode=0,seq=39353][2nd=39354]}, io_info.fd_={first_id:1152921504606886329, second_id:39354, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}) [2025-03-17 09:56:58.382388] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=31] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:57378}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634309} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:58.382428] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=26] end stmt(ret=0, tx_id=9634309, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf38b200, tx_id:{txid:9634309}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520162, assoc_session_id:3222520162, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176618342789, active_ts:1742176618342789, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648342783, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462457777599, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618323652}], exec_info_reap_ts:1742176618323651, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618323648, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222520162, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf38b200}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.382598] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634309}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176648342783, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618342789}{self_ls_id:{id:1}, session_id:3222520162, associated_session_id:3222520162, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634309}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462457777599, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618342789, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634309} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.382671] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=65] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634309}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176648342783, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618342789}{self_ls_id:{id:1}, session_id:3222520162, associated_session_id:3222520162, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634309}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462457777599, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618342789, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634309} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.382725] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=50] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634309}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176618382594, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:58.382745] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=18] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf38b200, tx_id:{txid:9634309}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520162, assoc_session_id:3222520162, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176618342789, active_ts:1742176618342789, commit_ts:-1, finish_ts:1742176618382085, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648342783, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462457777599, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618323652}], exec_info_reap_ts:1742176618323651, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618323648, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:58.382835] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F8-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:58.383072] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:229) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=23] successfully write small sstable(ret=0, block_info={nested_size:8192, nested_offset:1110016, macro_id:{[ver=1,mode=0,seq=39353][2nd=39354]}}, offset_=1118208, old_block=[]) [2025-03-17 09:56:58.383100] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162413516}], scn_array=[cnt:1, {val:1742176618154213002, v:0}], curr_size=122, accum_size=244, time_cost=24746) [2025-03-17 09:56:58.383284] INFO [STORAGE] build_single_node_tree (ob_index_block_builder.cpp:3582) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=30] successfully build single node tree, whose root is a data root(ret=0, block_desc={addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=316,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, buf:0x7f7adde710a0, height:1, is_meta_root:true}) [2025-03-17 09:56:58.383308] INFO [STORAGE] close_with_macro_seq (ob_index_block_builder.cpp:1666) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=19] succeed to close sstable index builder(res={root_desc:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=311,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, buf:0x7f7adde70560, height:2, is_meta_root:false}, data_root_desc:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=316,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, buf:0x7f7adde710a0, height:1, is_meta_root:true}, data_block_ids_.count():1, other_block_ids_.count():0, index_blocks_cnt:0, data_blocks_cnt:1, micro_block_cnt:1, data_column_cnt:47, data_column_checksums:[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], row_count:19, max_merged_trans_version:1741917093691416000, contain_uncommitted_row:false, occupy_size:3778, original_size:3145, data_checksum:432605713, use_old_macro_block_count:0, compressor_type:1, root_row_store_type:0, nested_offset:0, nested_size:2097152, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, root_macro_seq:0, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000"}) [2025-03-17 09:56:58.383360] INFO [STORAGE] init (ob_sstable.cpp:306) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=33] succeeded to init sstable(ret=0, this={ObITable:{this:0x7f798d468060, key:{tablet_id:{id:377}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741917602909677000, v:0}}}, ref_cnt:0, upper_trans_version:1741917093691416000, timestamp:0}, this:0x7f798d468060, addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=200,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:1110016, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:19, occupy_size:3778, data_checksum:432605713, max_merged_trans_version:1741917093691416000, upper_trans_version:1741917093691416000, filled_tx_scn:{val:1741917602909677000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:123, row_count_:19, occupy_size_:3778, original_size_:3145, data_checksum_:432605713, index_type_:0, rowkey_column_count_:5, column_cnt_:47, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278088813336, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741917093691416000, max_merged_trans_version_:1741917093691416000, recycle_version:1741629603950087000, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741917602909677000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:1, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=311,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f798d4683d0, block_data:{buf:0x7f798d468680, size:64, extra_buf:0x7f798d4686c0, extra_size:1191, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=316,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f798d468510, block_data:{buf:0x7f798d468510, size:316, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f798d468650, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1110016, nested_size:8192}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:false}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:true}) [2025-03-17 09:56:58.383447] INFO [STORAGE] set_upper_trans_version (ob_sstable.cpp:1030) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=86] finish set upper trans version(ret=0, key_={tablet_id:{id:377}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741917602909677000, v:0}}}, meta=0x7f798d468130, old_val=1741917093691416000, upper_trans_version=1741917093691416000, meta_cache={version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:8192, nested_offset:1110016, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:19, occupy_size:3778, data_checksum:432605713, max_merged_trans_version:1741917093691416000, upper_trans_version:1741917093691416000, filled_tx_scn:{val:1741917602909677000, v:0}, contain_uncommitted_row:false}) [2025-03-17 09:56:58.383470] INFO [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:736) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=22] successfully rebuild one sstable(ret=0, block_info={nested_size:8192, nested_offset:1110016, macro_id:{[ver=1,mode=0,seq=39353][2nd=39354]}}, new_sstable.get_key()={tablet_id:{id:377}, column_group_idx:0, table_type:"MINOR", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741917602909677000, v:0}}}) [2025-03-17 09:56:58.383897] INFO [STORAGE] init (ob_index_block_builder.cpp:571) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=15] init sstable index builder(ret=0, data_desc={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:47, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:47, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=26 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=27 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=29 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=42 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=45 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=46 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=47 {type:"TINYINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=48 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=49 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=50 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=51 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=52 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=53 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=54 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=55 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=56 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=57 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=58 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=59 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=60 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:1, micro_block_size:16384}, index_store_desc_={desc:{static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:true, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}}, container_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}, leaf_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:2095976}, &object_cleaner_=0x7f7acca40520) [2025-03-17 09:56:58.384060] INFO [STORAGE] init (ob_index_block_builder.cpp:3831) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=157] succeed to init ObIndexBlockRebuilder(task_idx=-1, task_type=2, device_handle_=NULL, index_store_desc_={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:0x7f7acca3a050, need_pre_warm:true, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}, container_store_desc={static_desc:{ls_id:{id:1}, tablet_id:{id:377}, tablet_transfer_seq:0, merge_type:"MINOR_MERGE", snapshot_version:1741991628854579000, end_scn:{val:1741991628854579000, v:0}, exec_mode:"EXEC_MODE_LOCAL", is_ddl:false, compressor_type:1, macro_block_size:2097152, macro_store_size:1887436, micro_block_size_limit:2095976, schema_version:1741278088813336, encrypt_id:0, master_key_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", major_working_cluster_version:0, micro_index_clustered:false, progressive_merge_round:1, need_submit_io:true, encoding_granularity:0}, row_store_type:"flat_row_store", col_desc:{is_row_store:true, table_cg_idx:0, row_column_count:6, rowkey_column_count:5, schema_rowkey_col_cnt:3, full_stored_col_cnt:47, col_desc_array:[cnt:6, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"binary", coercibility:"INVALID"} order=-1], default_col_checksum_array_valid:true, col_default_checksum_array:[cnt:47, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0], agg_meta_array:[cnt:0]}, encoder_opt:{enable_bit_packing:false, store_sorted_var_len_numbers_dict:false, enable_raw:false, enable_dict:false, enable_int_diff:false, enable_str_diff:false, enable_hex_pack:false, enable_rle:false, enable_const:false}, sstable_index_builder:null, need_pre_warm:false, need_build_hash_index_for_micro_block:false, data_store_type:6, micro_block_size:16384}) [2025-03-17 09:56:58.384593] INFO [STORAGE.BLKMGR] async_write (ob_macro_block_handle.cpp:219) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=96] Async write macro block(macro_id_={[ver=1,mode=0,seq=39353][2nd=39354]}, io_info.fd_={first_id:1152921504606886329, second_id:39354, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}) [2025-03-17 09:56:58.394367] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40303021260, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:58.395175] INFO [STORAGE.BLKMGR] write_block (ob_shared_macro_block_manager.cpp:229) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=30] successfully write small sstable(ret=0, block_info={nested_size:32768, nested_offset:1118208, macro_id:{[ver=1,mode=0,seq=39353][2nd=39354]}}, offset_=1150976, old_block=[]) [2025-03-17 09:56:58.395206] EDIAG [STORAGE] deserialize (ob_macro_block_common_header.cpp:133) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4034] deserialize error(ret=-4034, *this={header_size:0, version:0, magic:0, attr:0, payload_size:0, payload_checksum:0}) BACKTRACE:0x810107f 0x7b89ba6 0x7ee92d4 0x7ee8ac6 0x7ee8768 0x7ee8bfb 0x1791fc2e 0x17a77792 0x17a79468 0x185d7f45 0x185df53c 0x7b75587 0x7b27c01 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.395241] WDIAG [STORAGE] get_meta_block (ob_index_block_builder.cpp:4027) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4034] fail to deserialize common header(ret=-4034, buf=0x7f79fecbc050, buf_size=32768, pos=0) [2025-03-17 09:56:58.395253] WDIAG [STORAGE] inner_get_macro_meta (ob_index_block_builder.cpp:3891) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=11][errcode=-4034] fail to get meta block and read info(ret=-4034, buf=0x7f79fecbc050, size=32768) [2025-03-17 09:56:58.395260] WDIAG [STORAGE] append_macro_row (ob_index_block_builder.cpp:3981) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=6][errcode=-4034] fail to get macro meta(ret=-4034, macro_id={[ver=1,mode=0,seq=39353][2nd=39354]}) [2025-03-17 09:56:58.395270] WDIAG [STORAGE.BLKMGR] rebuild_sstable (ob_shared_macro_block_manager.cpp:717) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4034] fail to append macro row(ret=-4034, block_info={nested_size:32768, nested_offset:1118208, macro_id:{[ver=1,mode=0,seq=39353][2nd=39354]}}) [2025-03-17 09:56:58.395287] WDIAG [STORAGE.BLKMGR] update_tablet (ob_shared_macro_block_manager.cpp:625) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4034] fail to rebuild sstable and update tablet(ret=-4034) [2025-03-17 09:56:58.395302] WDIAG [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:502) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4034] fail to update tablet(ret=-4034, tablet_handle={obj:0x7f7af325e0a0, obj_pool:0x7f7b311cfd70, allocator:null, wash_priority:1, allow_copy_and_assign:true}, macro_ids=[cnt:11, {[ver=1,mode=0,seq=20279][2nd=20280]}, {[ver=1,mode=0,seq=32444][2nd=32445]}, {[ver=1,mode=0,seq=29474][2nd=29475]}, {[ver=1,mode=0,seq=23849][2nd=23850]}, {[ver=1,mode=0,seq=13287][2nd=13288]}, {[ver=1,mode=0,seq=32476][2nd=32477]}, {[ver=1,mode=0,seq=20248][2nd=20249]}, {[ver=1,mode=0,seq=8781][2nd=8782]}, {[ver=1,mode=0,seq=29410][2nd=29411]}, {[ver=1,mode=0,seq=20221][2nd=20222]}, {[ver=1,mode=0,seq=13236][2nd=13237]}]) [2025-03-17 09:56:58.395352] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:765) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=32] tablet skip filter destructed(total_cnt=204, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=15851311, total_skip_cnt=45934, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2025-03-17 09:56:58.395368] WDIAG [STORAGE.BLKMGR] runTimerTask (ob_shared_macro_block_manager.cpp:936) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4034] fail to defragment small sstables(ret=-4034) [2025-03-17 09:56:58.415406] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.415645] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.415669] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.415961] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.416023] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.416130] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.416218] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.428390] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859687638}], scn_array=[cnt:1, {val:1742176618393851000, v:0}], curr_size=588, accum_size=588, time_cost=33684) [2025-03-17 09:56:58.432525] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=41] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:58.435897] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=15] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:56:58.436619] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=22] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:56:58.436688] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=18] inner load succ(latest_freeze_info={frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615071995000, v:0}, frozen_statuses_.count():3, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629601710007000, v:0}, schema_version:1741597433156656, data_version:17180067072}, {frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}]}) [2025-03-17 09:56:58.445694] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=10] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:58.447340] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=19][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.447376] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=35][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.447396] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=19][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.447406] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6c78, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618446910, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:387, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.447476] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=60][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6c78, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618446910, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:387, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:1742176618447469}}, part_id:-1}) [2025-03-17 09:56:58.447722] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=31][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.447741] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=18][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.447747] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79f029915a, src_buf_size=1611) [2025-03-17 09:56:58.447756] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c51c0, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176618447494, enqueue_used:2, dequeue_used:15, submit_used:1, return_used:188, callback_enqueue_used:2, callback_dequeue_used:18, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.447789] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=29][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c51c0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176618447494, enqueue_used:2, dequeue_used:15, submit_used:1, return_used:188, callback_enqueue_used:2, callback_dequeue_used:18, callback_finish_used:-1, end_used:1742176618447786}}, part_id:-1}) [2025-03-17 09:56:58.447925] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222519947, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3382a0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:58.447976] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=36] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634313} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:58.448011] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=30] end stmt(ret=0, tx_id=9634313, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3382a0, tx_id:{txid:9634313}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519947, assoc_session_id:3222519947, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176618446080, active_ts:1742176618446080, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648446074, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462561061269, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618443956}], exec_info_reap_ts:1742176618443955, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618443952, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222519947, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3382a0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.448185] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634313}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176648446074, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618446080}{self_ls_id:{id:1}, session_id:3222519947, associated_session_id:3222519947, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634313}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462561061269, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618446080, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634313} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.448273] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=72] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634313}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176648446074, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618446080}{self_ls_id:{id:1}, session_id:3222519947, associated_session_id:3222519947, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634313}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462561061269, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618446080, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634313} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.448344] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=64] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634313}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176618448180, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:58.448368] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=21] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3382a0, tx_id:{txid:9634313}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519947, assoc_session_id:3222519947, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176618446080, active_ts:1742176618446080, commit_ts:-1, finish_ts:1742176618448210, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648446074, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462561061269, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618443956}], exec_info_reap_ts:1742176618443955, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618443952, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:58.448483] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633F9-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:58.451847] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=6) [2025-03-17 09:56:58.465582] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=38] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}]) [2025-03-17 09:56:58.476357] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=33] tx gc loop thread is running(MTL_ID()=1) [2025-03-17 09:56:58.476396] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=37] try gc retain ctx [2025-03-17 09:56:58.476403] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] try to retry start_working [2025-03-17 09:56:58.476437] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=4] finish update min start scn(max_decided_scn={val:1742176618344925000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176608420054001, v:0}, keep_alive_scn_:{val:1742176613453531001, v:0}, update_ts_:1742176618475901}) [2025-03-17 09:56:58.477061] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176626588679000, latest_log_ts:{val:1742176609930802001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:56:58.479283] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162413638}], scn_array=[cnt:1, {val:1742176618154213003, v:0}], curr_size=122, accum_size=366, time_cost=20868) [2025-03-17 09:56:58.479364] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701493}], scn_array=[cnt:1, {val:1742176618154213003, v:0}], curr_size=122, accum_size=244, time_cost=20965) [2025-03-17 09:56:58.479722] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.481309] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=31] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:58.481340] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=27] temporary file module use no blocks [2025-03-17 09:56:58.481348] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=6] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:58.481360] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:58.481369] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=19] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:58.481374] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863113373066, flush_io_finished_ret_:0, flush_io_finished_round_:859376, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:58.481398] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=26] temporary file module use no blocks [2025-03-17 09:56:58.481405] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:58.481415] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=10] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:58.481431] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=16] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863113373151, flush_io_finished_ret_:0, flush_io_finished_round_:580426, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:58.481450] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=16] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:56:58.481471] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=18] temporary file module use no blocks [2025-03-17 09:56:58.481480] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=9] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:56:58.481489] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=8] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:56:58.481501] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=11] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863113373208, flush_io_finished_ret_:0, flush_io_finished_round_:580427, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:56:58.485028] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=24][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:352, tid:3539397}]) [2025-03-17 09:56:58.503134] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463507427}], scn_array=[cnt:1, {val:1742176618460984000, v:0}], curr_size=546, accum_size=3086, time_cost=41075) [2025-03-17 09:56:58.509804] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=1][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.509847] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=42][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.509875] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=27][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.509887] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6570, ref_cnt_:1, raw_buf_:0x7f7ac4e04150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0072-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618496688, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:13050, callback_enqueue_used:2, callback_dequeue_used:27, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.509937] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=43][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6570, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0072-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618496688, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:13050, callback_enqueue_used:2, callback_dequeue_used:27, callback_finish_used:-1, end_used:1742176618509932}}, part_id:-1}) [2025-03-17 09:56:58.509968] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.509943] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=32][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618496688, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:13050, callback_enqueue_used:2, callback_dequeue_used:27, callback_finish_used:163, end_used:-33}}) [2025-03-17 09:56:58.509979] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=10][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.509986] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=7][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.509984] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=38][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:58.509996] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=10][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:58.509990] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=4][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6be8, ref_cnt_:1, raw_buf_:0x7f7ac4e06170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE19-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618506283, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:3588, callback_enqueue_used:1, callback_dequeue_used:73, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.510018] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=17][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6be8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE19-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618506283, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:3588, callback_enqueue_used:1, callback_dequeue_used:73, callback_finish_used:-1, end_used:1742176618510009}}, part_id:-1}) [2025-03-17 09:56:58.510032] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=27][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618506283, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:3588, callback_enqueue_used:1, callback_dequeue_used:73, callback_finish_used:69, end_used:-27}}) [2025-03-17 09:56:58.510076] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=43][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:58.510087] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=9][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:58.510323] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.510345] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=22][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.510351] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaac31510, src_buf_size=295) [2025-03-17 09:56:58.510357] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7260, ref_cnt_:1, raw_buf_:0x7f7aaac30150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0072-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618510008, enqueue_used:1, dequeue_used:40, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.510391] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=9][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618510008, enqueue_used:1, dequeue_used:40, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:1742176618510381}}) [2025-03-17 09:56:58.510415] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=22][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:58.510389] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=22][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7260, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0072-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6b6c080, time_log:{begin_ts:1742176618510008, enqueue_used:1, dequeue_used:40, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:1742176618510381}}, part_id:-1}) [2025-03-17 09:56:58.510430] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=14][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac6b6c080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.510457] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=27][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac6b6c080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.510472] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=12][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:58.510483] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=11][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc5f450}) [2025-03-17 09:56:58.510499] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=52][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.510524] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=25][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.510501] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=17][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc601f8, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:0, rowkey_end_idx:-4294967296, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc601f8, micro_handle:NULL}) [2025-03-17 09:56:58.510530] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7ad7903510, src_buf_size=295) [2025-03-17 09:56:58.510532] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=30][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:56:58.510536] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c5d00, ref_cnt_:1, raw_buf_:0x7f7ad7902150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE19-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618510098, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:377, callback_enqueue_used:1, callback_dequeue_used:7, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.510569] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=22][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c5d00, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE19-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618510098, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:377, callback_enqueue_used:1, callback_dequeue_used:7, callback_finish_used:-1, end_used:1742176618510560}}, part_id:-1}) [2025-03-17 09:56:58.510577] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=9][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176618510098, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:377, callback_enqueue_used:1, callback_dequeue_used:7, callback_finish_used:91, end_used:-27}}) [2025-03-17 09:56:58.510594] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=17][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:56:58.510600] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=5][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a3f2b2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.510615] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=15][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a3f2b2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.510629] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=12][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:58.510639] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=10][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc5d610}) [2025-03-17 09:56:58.510538] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a96a20a58, local_datums:0x7f7a96a20a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:56:58.510667] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=128][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a96a20a58, local_datums:0x7f7a96a20a58}, datums:[]}) [2025-03-17 09:56:58.510655] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=15][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc5e3b8, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:1073741840, rowkey_end_idx:1817, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc5e3b8, micro_handle:NULL}) [2025-03-17 09:56:58.510677] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=10][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176618460984000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:56:58.510683] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=27][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:56:58.510685] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=7][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7a96a1fac0) [2025-03-17 09:56:58.510691] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:56:58.510688] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=5][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f77ec014fa8, local_datums:0x7f77ec014fa8}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:56:58.510810] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=121][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f77ec014fa8, local_datums:0x7f77ec014fa8}, datums:[]}) [2025-03-17 09:56:58.510820] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=9][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176618460984000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:56:58.510825] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=4][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f77ec014010) [2025-03-17 09:56:58.510830] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=4][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:56:58.510697] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1003},{"BIGINT":1};{"BIGINT":1003},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176648395989, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7a96a1f3d0, valid:true, source:2, core:{version:{val:1742176618460984000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176618495080, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7a96a1b9a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7a96a1f6a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7a96a1b848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7a96a1fac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:56:58.510948] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=250][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:58.510956] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=7][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:56:58.510962] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:56:58.510970] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=7][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:56:58.510976] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:56:58.510981] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:56:58.510986] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a96a17b50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:56:58.510994] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=7][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:56:58.510998] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=3][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:56:58.511003] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:56:58.511007] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=3][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:56:58.511011] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=3][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a96a17b50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:56:58.511016] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:56:58.511021] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:58.511026] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:56:58.511031] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}) [2025-03-17 09:56:58.511038] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] execute failed(ret=-4016, tenant_id=1, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741661297536408) [2025-03-17 09:56:58.511046] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=7][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:58.511072] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=8] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8558a0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519223, assoc_session_id:3222519223, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176618496150, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176618495080, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222519223, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace8558a0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.511122] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=42][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:58.511128] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:58.511134] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:58.510835] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=4][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1001},{"BIGINT":1};{"BIGINT":1001},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176648404599, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f77ec013920, valid:true, source:2, core:{version:{val:1742176618460984000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7ad2ac3da0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7ad2ac3dc0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7ad2ac3d60, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7ad2ac3e10, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176618504660, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7ad2ac3da0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7ad2ac3dc0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7ad2ac3d60, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7ad2ac3e10, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f77ec00ff50, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f77ec013bf0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], op_filters:null, row2exprs_projector:0x7f77ec00fdf8, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f77ec014010, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:56:58.511165] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=329][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:58.511175] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=9][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:56:58.511175] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=32][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:58.511183] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=8][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:56:58.511189] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=11][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, aret=-4016, ret=-4016) [2025-03-17 09:56:58.511191] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=7][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:56:58.511199] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:1806) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=6][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:56:58.511199] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=8][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:56:58.511205] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1) [2025-03-17 09:56:58.511211] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1746) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] execute_read failed(ret=-4016, cluster_id=1741272511, tenant_id=1) [2025-03-17 09:56:58.511216] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:138) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] query failed(ret=-4016, conn=0x7f7aed35a3a0, start=1742176618496578, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:56:58.511207] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=7][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:56:58.511222] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:67) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=5][errcode=-4016] read failed(ret=-4016) [2025-03-17 09:56:58.511228] WDIAG [SHARE] exec_read (ob_ls_i_life_manager.h:213) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=4][errcode=-4016] failed to read(ret=-4016, ret="OB_ERR_UNEXPECTED", exec_tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:56:58.511226] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:58.511226] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=18][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f77ec00c100}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:56:58.511244] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=18][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:56:58.511255] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=11][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:56:58.511265] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=9][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:56:58.511277] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=59] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:58.511291] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=25][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:56:58.511299] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=8][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f77ec00c100}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:56:58.511306] WDIAG [SHARE] get_ls_primary_zone_info (ob_ls_status_operator.cpp:1062) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0072-0-0] [lt=13][errcode=-4016] failed to read ls recovery(ret=-4016, ret="OB_ERR_UNEXPECTED", tenant_id=1003, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:56:58.511310] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=11][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:56:58.511320] WDIAG [SQL.EXE] sync_send_result (ob_remote_executor_processor.cpp:286) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=10][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:56:58.511328] WDIAG [SQL.EXE] execute_remote_plan (ob_remote_executor_processor.cpp:485) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=8][errcode=-4016] sync send result failed(ret=-4016) [2025-03-17 09:56:58.511354] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=11] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6adf220, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222496794, assoc_session_id:3222496794, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176618505719, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176618504660, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ad1c721f8, id:3222496794, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6adf220}, exec_ctx.get_errcode()=0) [2025-03-17 09:56:58.511410] WDIAG [SQL.EXE] execute_with_sql (ob_remote_executor_processor.cpp:716) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE19-0-0] [lt=50][errcode=-4016] execute remote plan failed(ret=-4016, task={tenant_schema_version:1741661297536408, sys_schema_version:1741661297536408, runner_svr:"192.168.1.62:2882", ctrl_svr:"192.168.1.64:2882", task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.64:2882", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:18446744073709551615}, task_id:0, task_cnt:0}, remote_sql_info:{use_ps:true, is_batched_stmt:false, is_original_ps_mode:false, ps_param_cnt:2, remote_sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = ? and a.tenant_id = ?", ps_params:[{obj:{"BIGINT":1}, accuracy:{length:1, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}, {obj:{"BIGINT":1001}, accuracy:{length:4, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}]}, snapshot:{this:0x7f77ec004208, valid:false, source:0, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}}, exec_ctx_.get_das_ctx().get_snapshot()={this:0x7f77ec0094a0, valid:true, source:2, core:{version:{val:1742176618460984000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}) [2025-03-17 09:56:58.520796] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570F-0-0] [lt=14][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=325, pos=295) [2025-03-17 09:56:58.521340] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570F-0-0] [lt=26][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=63, pos=48) [2025-03-17 09:56:58.521965] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F570F-0-0] [lt=12][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=80, len=328, pos=298) [2025-03-17 09:56:58.533772] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=57] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=26731, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:56:58.533812] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859688226}], scn_array=[cnt:1, {val:1742176618393851001, v:0}], curr_size=122, accum_size=710, time_cost=24859) [2025-03-17 09:56:58.533880] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8124) [2025-03-17 09:56:58.543996] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=34] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:56:58.544033] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=29] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:56:58.544046] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=11] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=11748) [2025-03-17 09:56:58.551674] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:261) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1002) [2025-03-17 09:56:58.551701] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=22][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.551721] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.551738] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552240] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=4][errcode=-5150] switch tenant fail(tenant_id=1001, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552267] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552525] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552551] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-5150] switch tenant fail(tenant_id=1005, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552569] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.552586] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:58.553053] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:194) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=8] ls blacklist refresh finish(cost_time=16074) [2025-03-17 09:56:58.562890] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:58.562950] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=7] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:58.579787] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.580107] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701615}], scn_array=[cnt:1, {val:1742176618154213004, v:0}], curr_size=122, accum_size=366, time_cost=21619) [2025-03-17 09:56:58.594711] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40300924108, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:58.595315] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [117795][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=26] LogLoopThread round_cost_time(us)(round_cost_time=13) [2025-03-17 09:56:58.597990] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=34] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1742176618393851000, v:0}, new_max_version={val:1742176618393851000, v:0}, last_min_version={val:1742176618193719000, v:0}, last_max_version={val:1742176618193719000, v:0}, total_time_us=4475, delta_us=204137) [2025-03-17 09:56:58.598021] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1742176618393851000, v:0}, min_version={val:1742176618393851000, v:0}, max_version={val:1742176618393851000, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:56:58.599048] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=12] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:58.601340] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=36][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.601379] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=38][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.601398] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.601407] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8d18, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FA-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618600933, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.601472] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=57][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8d18, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FA-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618600933, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:-1, end_used:1742176618601467}}, part_id:-1}) [2025-03-17 09:56:58.601481] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=70][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618600933, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:167, end_used:-37}}) [2025-03-17 09:56:58.601518] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=36][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:58.601529] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:58.601778] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.601799] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=20][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.601806] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a0e5bd15a, src_buf_size=1611) [2025-03-17 09:56:58.601812] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7c38, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FA-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618601544, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:180, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:58.601853] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7c38, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FA-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618601544, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:180, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176618601846}}, part_id:-1}) [2025-03-17 09:56:58.601865] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=11][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176618601544, enqueue_used:1, dequeue_used:33, submit_used:1, return_used:180, callback_enqueue_used:1, callback_dequeue_used:17, callback_finish_used:-1, end_used:1742176618601846}}) [2025-03-17 09:56:58.601882] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=17][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:58.601888] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=5][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a2293c080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.601907] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=19][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a2293c080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:58.601923] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=14][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:58.601937] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=14][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:58.601962] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=24][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:58.602012] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=50][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:58.602018] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=5][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:58.602024] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=5][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:58.602034] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:58.602041] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176648500755, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:58.602095] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=54][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176648500755, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a091c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: CFD27C1A80300600, int: 1742176618599119}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}}) [2025-03-17 09:56:58.602140] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=43][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a091c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: CFD27C1A80300600, int: 1742176618599119}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:58.602186] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=45][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:58.602193] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:58.602201] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}, ret=-4016) [2025-03-17 09:56:58.602234] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=9][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f79f02a0e50, local_datums:0x7f79f02a0aa0}, datums:[col_id=0:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a091c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a092d, hex: CFD27C1A80300600, int: 1742176618599119}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a0935, hex: CFD27C1A80300600, int: 1742176618599119}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a093d, hex: CFD27C1A80300600, int: 1742176618599119}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a0945, hex: 0A00000000000000, int: 10}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a094d, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f79f02a0955, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a095b, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 36, flag: 0, null: 0, ptr: 0x7f79f02a096c, hex: 44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829, cstr: DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f79f02a0990, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a0999, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09a1, hex: 00C40FE7FF2F0600, int: 1741626000000000}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a09a9, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f79f02a09ba, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176648500755, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176648500755, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176618593946000, v:0}, tx_id:{txid:9634316}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634316}, tx_desc:{this:0x7f7acf2b8b00, tx_id:{txid:9634316}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519892, assoc_session_id:3222519892, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176618600287, active_ts:1742176618600287, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:1742176648600277, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462714988222, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176618595014}], exec_info_reap_ts:1742176618595013, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618595010, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:58.602314] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=79][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:58.602322] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a091c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a092d, hex: CFD27C1A80300600, int: 1742176618599119}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a091c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a092d, hex: CFD27C1A80300600, int: 1742176618599119}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:58.599119"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176648500755, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:58.602373] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=50][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:58.602380] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:58.602391] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:58.602398] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:58.602411] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=9][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:58.602441] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=29][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:58.602445] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:58.602452] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:58.602460] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176648600755, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388449, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf2b8b00, tx_id:{txid:9634316}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519892, assoc_session_id:3222519892, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176618600287, active_ts:1742176618600287, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:1742176648600277, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462714988222, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176618595014}], exec_info_reap_ts:1742176618595013, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618595010, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634316}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176648500755, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176648500755, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176648500755, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:58.602955] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=495][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:58.602961] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=5][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:58.602969] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:58.602973] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=3][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:58.602977] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:58.602982] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:58.602991] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:58.602998] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:58.603005] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:58.603012] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:58.603019] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:58.603024] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:58.603030] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}) [2025-03-17 09:56:58.603039] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=9][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:58.603049] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=9][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:58.603065] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=10] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222519892, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2b8b00}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:58.603093] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634316} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:58.603117] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=20] end stmt(ret=0, tx_id=9634316, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2b8b00, tx_id:{txid:9634316}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519892, assoc_session_id:3222519892, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176618600287, active_ts:1742176618600287, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:1742176648600277, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462714988222, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618595014}], exec_info_reap_ts:1742176618595013, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618595010, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222519892, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2b8b00}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.603159] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=41][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:58.603164] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:58.603168] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=4][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:58.603179] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:58.603188] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:58.603196] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=6][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')) [2025-03-17 09:56:58.603211] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:58.603219] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), is_user_sql=false) [2025-03-17 09:56:58.603227] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176618599119), usec_to_time(1742176618599119), 10, usec_to_time(1742176618599119), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), conn=0x7f7ac70b2050) [2025-03-17 09:56:58.603235] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:58.603291] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634316}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176648600277, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618600287}{self_ls_id:{id:1}, session_id:3222519892, associated_session_id:3222519892, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634316}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462714988222, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618600287, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634316} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.603346] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=45] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634316}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176648600277, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618600287}{self_ls_id:{id:1}, session_id:3222519892, associated_session_id:3222519892, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634316}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462714988222, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618600287, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634316} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.603383] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=33] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634316}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176618603287, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:58.603397] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=13] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2b8b00, tx_id:{txid:9634316}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519892, assoc_session_id:3222519892, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176618600287, active_ts:1742176618600287, commit_ts:-1, finish_ts:1742176618602435, timeout_us:29999990, lock_timeout_us:-1, expire_ts:1742176648600277, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462714988222, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618595014}], exec_info_reap_ts:1742176618595013, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618595010, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:58.603479] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=32][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:58.603487] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=8][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:58.603495] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FA-0-0] [lt=7][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:58.611035] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=37] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2025-03-17 09:56:58.611067] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=29] start lib cache evict(tenant_id=1004, mem_hold=652214272, mem_limit=773094110, cache_obj_num=1799, cache_node_num=1718) [2025-03-17 09:56:58.611077] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=9] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=652214272, mem_limit=773094110, cache_obj_num=1799, cache_node_num=1718) [2025-03-17 09:56:58.611088] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=10] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:58.614257] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=14] Dumping All Cache Objs(alloc_obj_list.count()=1799, alloc_obj_list=[{obj_id:46, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179056}, {obj_id:47, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179800}, {obj_id:48, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:49, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:54, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:206, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180416}, {obj_id:207, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184298}, {obj_id:213, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228134}, {obj_id:214, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234205}, {obj_id:217, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207208}, {obj_id:220, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203337}, {obj_id:222, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197116}, {obj_id:223, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220151}, {obj_id:225, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208334}, {obj_id:226, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194033}, {obj_id:227, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190716}, {obj_id:228, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189404}, {obj_id:229, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204465}, {obj_id:230, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198372}, {obj_id:232, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189580}, {obj_id:233, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192620}, {obj_id:234, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189612}, {obj_id:235, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192621}, {obj_id:236, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189548}, {obj_id:237, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196031}, {obj_id:238, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191118}, {obj_id:239, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187276}, {obj_id:240, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214929}, {obj_id:241, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187052}, {obj_id:242, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192050}, {obj_id:243, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187452}, {obj_id:244, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195907}, {obj_id:245, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187148}, {obj_id:246, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188348}, {obj_id:247, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195840}, {obj_id:248, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190558}, {obj_id:258, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:260, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184937}, {obj_id:261, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:264, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190171}, {obj_id:265, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:269, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176624}, {obj_id:271, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198826}, {obj_id:274, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:275, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189141}, {obj_id:276, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195949}, {obj_id:277, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190297}, {obj_id:278, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:279, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199077}, {obj_id:280, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:281, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179904}, {obj_id:284, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179416}, {obj_id:285, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:286, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:287, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178272}, {obj_id:288, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:289, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182249}, {obj_id:290, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:292, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:293, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226000}, {obj_id:294, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:295, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:296, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:297, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:298, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177072}, {obj_id:299, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177088}, {obj_id:305, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180218}, {obj_id:307, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189628}, {obj_id:308, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190140}, {obj_id:309, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161272}, {obj_id:310, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:311, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:312, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169842}, {obj_id:313, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165668}, {obj_id:314, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160523}, {obj_id:315, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:320, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:323, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188615}, {obj_id:324, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178480}, {obj_id:331, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179771}, {obj_id:333, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:341, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:343, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:344, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297040}, {obj_id:346, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268773}, {obj_id:349, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176557}, {obj_id:350, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178156}, {obj_id:351, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248983}, {obj_id:354, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202125}, {obj_id:355, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:359, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:252380}, {obj_id:367, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:380, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180292}, {obj_id:385, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:386, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:393, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:400, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:402, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182198}, {obj_id:454, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195541}, {obj_id:463, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160437}, {obj_id:464, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:475, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:476, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:477, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181131}, {obj_id:499, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:510, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233226}, {obj_id:723, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198826}, {obj_id:724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164191}, {obj_id:859, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:875, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180552}, {obj_id:878, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194268}, {obj_id:892, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195493}, {obj_id:894, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:900, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210468}, {obj_id:902, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195461}, {obj_id:907, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202507}, {obj_id:967, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:14054}, {obj_id:968, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1234581}, {obj_id:969, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183441}, {obj_id:970, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248351}, {obj_id:973, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232386}, {obj_id:974, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209724}, {obj_id:976, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185692}, {obj_id:977, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183053}, {obj_id:981, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:279851}, {obj_id:984, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235899}, {obj_id:985, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:986, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235514}, {obj_id:993, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236395}, {obj_id:997, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209553}, {obj_id:998, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:384189}, {obj_id:1003, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:346943}, {obj_id:1004, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1036860}, {obj_id:1005, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214841}, {obj_id:1013, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180726}, {obj_id:1017, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179887}, {obj_id:1021, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197096}, {obj_id:1022, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322960}, {obj_id:1025, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200656}, {obj_id:1026, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340153}, {obj_id:1027, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322500}, {obj_id:1031, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:434216}, {obj_id:1033, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:328057}, {obj_id:1036, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206700}, {obj_id:1037, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192229}, {obj_id:1038, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283899}, {obj_id:1048, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1049, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1053, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163056}, {obj_id:1061, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203111}, {obj_id:1065, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242505}, {obj_id:1066, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207162}, {obj_id:1067, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195590}, {obj_id:1069, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187300}, {obj_id:1070, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185320}, {obj_id:1074, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176744}, {obj_id:1077, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187396}, {obj_id:1079, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:1080, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:1081, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1082, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:1083, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:1084, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:1085, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:1088, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1090, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191548}, {obj_id:1092, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:453315}, {obj_id:1105, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264567}, {obj_id:1111, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196613}, {obj_id:1115, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196661}, {obj_id:1117, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1218, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1221, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1223, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1226, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1229, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1232, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1235, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1238, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1242, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1245, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1247, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1251, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1254, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1256, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1260, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1263, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1265, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1269, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1272, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1274, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1278, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1281, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1283, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1334, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204728}, {obj_id:1337, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1340, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1342, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1344, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1346, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1348, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1350, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1352, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1355, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1358, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1360, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1363, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1366, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1368, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1371, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1374, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1376, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1379, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1382, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1384, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1387, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1390, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1392, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1463, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196277}, {obj_id:1466, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196325}, {obj_id:1552, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197045}, {obj_id:1555, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197093}, {obj_id:1761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:1764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:1767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:1768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:1769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:1770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:1771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180844}, {obj_id:1772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181059}, {obj_id:1773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:1774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:1775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:1776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:1786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198904}, {obj_id:1787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331052}, {obj_id:1790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202496}, {obj_id:1791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348722}, {obj_id:1792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331352}, {obj_id:1799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178907}, {obj_id:1800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:1801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181537}, {obj_id:1802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182612}, {obj_id:1803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260440}, {obj_id:1804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230691}, {obj_id:1806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246344}, {obj_id:1808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223882}, {obj_id:1809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:448756}, {obj_id:1818, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:1820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:1823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1825, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:244350}, {obj_id:1871, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250368}, {obj_id:1872, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207905}, {obj_id:1873, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180710}, {obj_id:1874, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184839}, {obj_id:1875, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182239}, {obj_id:1880, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268450}, {obj_id:1881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214951}, {obj_id:1882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:659525}, {obj_id:1887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272054}, {obj_id:1888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232022}, {obj_id:1889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:676458}, {obj_id:1894, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206106}, {obj_id:1895, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:255666}, {obj_id:1896, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:364093}, {obj_id:1897, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:349058}, {obj_id:1901, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248939}, {obj_id:1905, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220341}, {obj_id:1906, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208181}, {obj_id:1907, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:431472}, {obj_id:1908, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:419831}, {obj_id:1912, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222082}, {obj_id:1913, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:439892}, {obj_id:1918, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218531}, {obj_id:1919, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:422873}, {obj_id:1920, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:410978}, {obj_id:1924, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185511}, {obj_id:1931, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:336934}, {obj_id:1932, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:295855}, {obj_id:1933, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1947, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204286}, {obj_id:1949, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:356276}, {obj_id:1950, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340211}, {obj_id:1959, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211545}, {obj_id:1964, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:301536}, {obj_id:1965, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260857}, {obj_id:1975, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201395}, {obj_id:1987, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184389}, {obj_id:2008, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239940}, {obj_id:2010, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:525534}, {obj_id:2016, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218641}, {obj_id:2022, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191835}, {obj_id:2023, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297162}, {obj_id:2024, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278315}, {obj_id:2035, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207891}, {obj_id:2036, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375309}, {obj_id:2037, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:357951}, {obj_id:2058, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193588}, {obj_id:2059, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:305064}, {obj_id:2060, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:287152}, {obj_id:2081, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184971}, {obj_id:2109, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216802}, {obj_id:2110, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:413810}, {obj_id:2111, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:402141}, {obj_id:2129, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225681}, {obj_id:2130, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:459054}, {obj_id:2131, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:446413}, {obj_id:2171, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184227}, {obj_id:2289, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184631}, {obj_id:2376, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204811}, {obj_id:2395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221640}, {obj_id:2402, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239174}, {obj_id:2431, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:2435, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:2440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184354}, {obj_id:2441, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182588}, {obj_id:2442, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210883}, {obj_id:2444, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210950}, {obj_id:2446, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197694}, {obj_id:2448, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:2452, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233751}, {obj_id:2502, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188484}, {obj_id:2507, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179273}, {obj_id:2573, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185063}, {obj_id:2574, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196061}, {obj_id:2589, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180547}, {obj_id:2590, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180525}, {obj_id:2593, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176624}, {obj_id:2632, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213894}, {obj_id:2634, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180532}, {obj_id:2635, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180529}, {obj_id:2651, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186780}, {obj_id:2652, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181185}, {obj_id:2653, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182472}, {obj_id:2654, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178176}, {obj_id:2655, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181137}, {obj_id:2656, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181817}, {obj_id:2677, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195622}, {obj_id:2678, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2679, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2680, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2681, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160496}, {obj_id:2682, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161287}, {obj_id:2686, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208041}, {obj_id:2688, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184389}, {obj_id:2690, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191552}, {obj_id:2693, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2714, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236571}, {obj_id:2743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163054}, {obj_id:2761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183342}, {obj_id:2765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223598}, {obj_id:2773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223592}, {obj_id:2814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:298681}, {obj_id:2827, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2835, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2836, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2837, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179776}, {obj_id:2838, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185454}, {obj_id:2952, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211463}, {obj_id:2953, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203185}, {obj_id:2954, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274521}, {obj_id:3013, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272510}, {obj_id:3108, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224378}, {obj_id:3165, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224762}, {obj_id:3186, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192084}, {obj_id:3187, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203098}, {obj_id:3231, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225178}, {obj_id:3390, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206653}, {obj_id:3391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186851}, {obj_id:3392, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190051}, {obj_id:3394, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192545}, {obj_id:3395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190052}, {obj_id:3396, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184534}, {obj_id:3397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190152}, {obj_id:3408, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3409, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208648}, {obj_id:3412, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221865}, {obj_id:3413, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226011}, {obj_id:3414, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185340}, {obj_id:3415, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191637}, {obj_id:3416, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:342020}, {obj_id:3417, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428864}, {obj_id:3428, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185714}, {obj_id:3429, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188924}, {obj_id:3436, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194201}, {obj_id:3437, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245052}, {obj_id:3438, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296079}, {obj_id:3439, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225978}, {obj_id:3440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196807}, {obj_id:3441, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197829}, {obj_id:3442, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296075}, {obj_id:3443, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191830}, {obj_id:3444, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197877}, {obj_id:3445, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189148}, {obj_id:3446, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3447, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3456, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3457, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3466, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3467, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3470, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3472, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3481, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182711}, {obj_id:3482, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189007}, {obj_id:3483, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3485, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3488, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3490, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3495, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3501, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3527, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190940}, {obj_id:3531, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210225}, {obj_id:3538, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196295}, {obj_id:3560, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264475}, {obj_id:3635, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199341}, {obj_id:3644, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195619}, {obj_id:3645, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160477}, {obj_id:3646, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161284}, {obj_id:3656, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3657, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3661, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3685, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:3693, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3696, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3699, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3702, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3705, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3708, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3711, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3713, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:3730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258706}, {obj_id:3732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200264}, {obj_id:3735, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3738, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223626}, [2025-03-17 09:56:58.616231] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=1972] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:56:58.628673] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=9] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=29116416) [2025-03-17 09:56:58.628766] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=45] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=94, size_used=0, mem_used=0) [2025-03-17 09:56:58.628823] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=26] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:58.635286] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.635356] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.635393] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.635737] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636063] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=28] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636064] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636192] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636229] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636411] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=20] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.636600] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:58.641415] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=31] gc stale ls task succ [2025-03-17 09:56:58.651949] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=23] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=5) [2025-03-17 09:56:58.655542] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=23] ts response task statistics(alloc_count=16165320, free_count=8082660) [2025-03-17 09:56:58.658474] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:58.658502] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:58.658529] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:58.658537] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:58.672634] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162413882}], scn_array=[cnt:1, {val:1742176618655657000, v:0}], curr_size=122, accum_size=610, time_cost=14081) [2025-03-17 09:56:58.672755] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=14494, avg_log_submit_to_slide_time=14510) [2025-03-17 09:56:58.672801] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:58.673980] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=26] gc stale ls task succ [2025-03-17 09:56:58.677602] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701737}], scn_array=[cnt:1, {val:1742176618655657000, v:0}], curr_size=122, accum_size=488, time_cost=19116) [2025-03-17 09:56:58.679854] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=45][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.686339] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:817) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=33] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}) [2025-03-17 09:56:58.709461] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=7] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:58.710469] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=37] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.220209898948669434e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=610, Throttle Trigger(MB)=366, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.278100362674315846e+02) [2025-03-17 09:56:58.711402] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=31][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.711453] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=50][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.711474] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=21][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:58.711497] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=43][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:58.711940] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222519175, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf224fa0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:58.711984] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634317} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:58.712013] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=26] end stmt(ret=0, tx_id=9634317, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf224fa0, tx_id:{txid:9634317}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519175, assoc_session_id:3222519175, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176618709814, active_ts:1742176618709814, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648709808, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462824915871, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618708756}], exec_info_reap_ts:1742176618708755, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618708752, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222519175, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf224fa0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.712133] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634317}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176648709808, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618709814}{self_ls_id:{id:1}, session_id:3222519175, associated_session_id:3222519175, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634317}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462824915871, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618709814, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634317} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.712226] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=83] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634317}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176648709808, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176618709814}{self_ls_id:{id:1}, session_id:3222519175, associated_session_id:3222519175, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634317}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409462824915871, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176618709814, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634317} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:58.712285] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=54] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634317}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176618712129, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:58.712304] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=17] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf224fa0, tx_id:{txid:9634317}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519175, assoc_session_id:3222519175, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176618709814, active_ts:1742176618709814, commit_ts:-1, finish_ts:1742176618711926, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176648709808, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409462824915871, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176618708756}], exec_info_reap_ts:1742176618708755, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176618708752, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:58.712392] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FC-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:58.733343] INFO [ARCHIVE] do_thread_task_ (ob_archive_fetcher.cpp:312) [117824][T1_ArcFetcher][T1][YB420A0B093E-00062FB7EE854125-0-0] [lt=33] ObArchiveFetcher is running(thread_index=0) [2025-03-17 09:56:58.737955] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [3539354][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=43] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2025-03-17 09:56:58.746360] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=4625) [2025-03-17 09:56:58.762447] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=22][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859689524}], scn_array=[cnt:1, {val:1742176618731136000, v:0}], curr_size=588, accum_size=2474, time_cost=30585) [2025-03-17 09:56:58.762765] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=54][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463508803}], scn_array=[cnt:1, {val:1742176618746046000, v:0}], curr_size=587, accum_size=4503, time_cost=16397) [2025-03-17 09:56:58.777058] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872701859}], scn_array=[cnt:1, {val:1742176618655657001, v:0}], curr_size=122, accum_size=610, time_cost=18498) [2025-03-17 09:56:58.778453] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=35] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:58.779936] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.781083] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=57][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414004}], scn_array=[cnt:1, {val:1742176618655657001, v:0}], curr_size=122, accum_size=732, time_cost=22505) [2025-03-17 09:56:58.795021] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] get number(ret=0, service_type_=0, range=1, base_id=1742176618794069000, start_id=1742176618794069000, end_id=1742176618794069001) [2025-03-17 09:56:58.795188] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40319798476, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:58.799925] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [3539266][T1004_DBMSSched][T1004][YB420A0B093E-00062FB7FC7B6B02-0-0] [lt=21] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:56:58.852078] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=35] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=4) [2025-03-17 09:56:58.852983] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=55][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859690112}], scn_array=[cnt:1, {val:1742176618794069000, v:0}], curr_size=588, accum_size=3062, time_cost=57847) [2025-03-17 09:56:58.858599] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:56:58.858646] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:56:58.858804] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=15] QSched: |cfg: root:1,x,0 net_in:1,1249M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:352K/68:14845 disk:352K/68:14845 t1m2:164K/24:14051 t1m2g0:164K/24:14051 t1004m2:88K/21:16830 t1004m2g0:88K/21:16830 t1003m2:100K/23:13862 t1003m2g0:100K/23:13862 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:56:58.862547] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=46] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16775, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:56:58.862582] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:56:58.862601] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:56:58.862624] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16896, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:56:58.862638] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:56:58.871563] INFO [CLOG] run1 (ob_garbage_collector.cpp:1466) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] Garbage Collector is running(seq_=58022, gc_interval=10000000) [2025-03-17 09:56:58.871621] INFO [CLOG] construct_server_ls_map_for_member_list_ (ob_garbage_collector.cpp:1601) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=40] self is leader, skip it(ls->get_ls_id()={id:1}) [2025-03-17 09:56:58.871641] INFO [CLOG] gc_check_member_list_ (ob_garbage_collector.cpp:1559) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=13] gc_check_member_list_ cost time(ret=0, time_us=38) [2025-03-17 09:56:58.871651] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] execute_gc cost time(ret=0, time_us=0) [2025-03-17 09:56:58.873805] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.873839] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=33][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:58.873857] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:58.873867] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6a80, ref_cnt_:1, raw_buf_:0x7f7a3f2b2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FB8623C9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176618873242, enqueue_used:1, dequeue_used:20, submit_used:1, return_used:513, callback_enqueue_used:2, callback_dequeue_used:24, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.873916] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=40][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6a80, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FB8623C9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176618873242, enqueue_used:1, dequeue_used:20, submit_used:1, return_used:513, callback_enqueue_used:2, callback_dequeue_used:24, callback_finish_used:-1, end_used:1742176618873909}}, part_id:-1}) [2025-03-17 09:56:58.873933] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539413][T1003_GCCollect][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=5][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:58.874267] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:58.874299] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=31][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:58.874306] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaac31510, src_buf_size=295) [2025-03-17 09:56:58.874312] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6e28, ref_cnt_:1, raw_buf_:0x7f7aaac30150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FB8623C9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176618873976, enqueue_used:1, dequeue_used:14, submit_used:1, return_used:262, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:58.874341] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=23][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6e28, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FB8623C9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176618873976, enqueue_used:1, dequeue_used:14, submit_used:1, return_used:262, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176618874337}}, part_id:-1}) [2025-03-17 09:56:58.874396] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539413][T1003_GCCollect][T1][YB420A0B093E-00062FB7FB8623C9-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ad817cf60, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222520232, assoc_session_id:3222520232, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176618872986, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176618872985, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222520232, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ad817cf60}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:58.874536] INFO [CLOG] get_ls_status_from_table (ob_garbage_collector.cpp:1498) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=0] failed to get ls status info from table(ret=-4016, tenant_id=1003, ls_id={id:1}) [2025-03-17 09:56:58.874552] WDIAG [CLOG] gc_check_ls_status_ (ob_garbage_collector.cpp:1683) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=0][errcode=0] get_ls_status_ failed(tmp_ret=-4016, ls->get_ls_id()={id:1}) [2025-03-17 09:56:58.874562] INFO [CLOG] execute_gc_ (ob_garbage_collector.cpp:1833) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=6] execute_gc cost time(ret=0, time_us=0) [2025-03-17 09:56:58.874571] INFO [SERVER] handle (ob_safe_destroy_handler.cpp:240) [3539413][T1003_GCCollect][T1003][Y0-0000000000000000-0-0] [lt=7] ObSafeDestroyHandler start process [2025-03-17 09:56:58.880011] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.903094] INFO [PALF] inner_write_once_ (log_block_handler.cpp:410) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=28027278, this={dio_aligned_buf:{buf_write_offset:2568, buf_padding_size:1528, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:6814084, truncate_used_ts:2784356}, log_block_size:67108864, io_fd:{first_id:-1, second_id:154, third_id:-1, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333a080}}, aligned_buf_len=4096, aligned_block_offset=28024832, buf_len=122, total_write_size=63593, total_write_size_after_dio=638976, ob_pwrite_used_ts=2394419, count=141, each_pwrite_cost=16981) [2025-03-17 09:56:58.903160] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=18898, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:56:58.903178] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463509390}], scn_array=[cnt:1, {val:1742176618746046001, v:0}], curr_size=122, accum_size=4625, time_cost=26429) [2025-03-17 09:56:58.903201] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=4625) [2025-03-17 09:56:58.903221] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=18, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:56:58.903240] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=26] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=4625) [2025-03-17 09:56:58.903287] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=11, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:56:58.903303] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=21177, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:56:58.903322] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=41, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:56:58.909189] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=38] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=15, avg_log_batch_cnt=1, total_group_log_size=7115, avg_group_log_size=474) [2025-03-17 09:56:58.927683] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=37][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:56:58.927717] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=32][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:56:58.932778] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:451) [117808][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=18][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:56:58.932883] WDIAG read_multi_row (ob_table_access_helper.h:238) [117808][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:56:58.933796] INFO delete_row (ob_table_access_helper.h:339) [117808][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=11] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1742176618932894) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1742176618932894) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2025-03-17 09:56:58.938275] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=39] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312800790618, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:56:58.938345] INFO [LIB] do_thread_recycle (threads.cpp:163) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=32] recycle one thread(this=0x7f7b319af170, total=3, remain=2) [2025-03-17 09:56:58.938357] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=11] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312800790618, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:56:58.939029] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [117737][T1_Occam][T1][YB420A0B093E-00062FB7EFFFBFD2-0-0] [lt=23] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:56:58.939135] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [117737][T1_Occam][T1][YB420A0B093E-00062FB7EFFFBFD4-0-0] [lt=27] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:56:58.943885] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859690700}], scn_array=[cnt:1, {val:1742176618794069001, v:0}], curr_size=122, accum_size=3184, time_cost=34687) [2025-03-17 09:56:58.945931] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=37] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:56:58.945978] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:56:58.946941] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=26] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:56:58.950544] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=12] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:56:58.958662] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:56:58.959108] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463509512}], scn_array=[cnt:1, {val:1742176618945509000, v:0}], curr_size=587, accum_size=587, time_cost=13109) [2025-03-17 09:56:58.969921] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [117761][T1_Occam][T1][YB420A0B093E-00062FB860254125-0-0] [lt=26] ====== tenant freeze timer task ====== [2025-03-17 09:56:58.980079] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=31][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:58.980511] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872702103}], scn_array=[cnt:1, {val:1742176618655657003, v:0}], curr_size=122, accum_size=854, time_cost=21851) [2025-03-17 09:56:58.981905] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414248}], scn_array=[cnt:1, {val:1742176618655657003, v:0}], curr_size=122, accum_size=976, time_cost=23195) [2025-03-17 09:56:58.991661] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=9] set tenant work area memory(tenant_id=1003, wa_pctg=5, limit=157286400, ret=0) [2025-03-17 09:56:58.991783] INFO [SERVER] update_tenant_wa_percentage (ob_tenant_duty_task.cpp:243) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=31] set tenant work area memory(tenant_id=1004, wa_pctg=5, limit=773094110, ret=0) [2025-03-17 09:56:58.991966] INFO [STORAGE] runTimerTask (ob_tenant_memory_printer.cpp:32) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=26] === Run print tenant memory usage task === [2025-03-17 09:56:58.992018] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:102) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=22] ====== tenants memory info ====== === TENANTS MEMORY INFO === unmanaged_memory_size= 188,051,456 [TENANT_MEMORY] tenant_id= 500 mem_tenant_limit= 9,223,372,036,854,775,807 mem_tenant_hold= 1,030,615,040 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 508 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 40,738,816 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 509 mem_tenant_limit= 1,073,741,824 mem_tenant_hold= 23,961,600 kv_cache_mem= 0 [TENANT_MEMORY] tenant_id= 1 now= 1,742,176,618,991,305 active_memstore_used= 467,664,896 total_memstore_used= 463,470,592 total_memstore_hold= 463,470,592 memstore_freeze_trigger_limit= 515,396,060 memstore_limit= 2,576,980,360 mem_tenant_limit= 6,442,450,944 mem_tenant_hold= 5,366,005,760 max_mem_memstore_can_get_now= 2,913,550,336 memstore_alloc_pos= 4,410,310,656 memstore_frozen_pos= 3,942,645,760 memstore_reclaimed_pos= 3,946,840,064 [TENANT_MEMORY] tenant_id= 1,003 now= 1,742,176,618,991,305 active_memstore_used= 58,720,256 total_memstore_used= 56,623,104 total_memstore_hold= 56,623,104 memstore_freeze_trigger_limit= 106,917,640 memstore_limit= 687,194,760 mem_tenant_limit= 1,717,986,918 mem_tenant_hold= 1,424,756,736 max_mem_memstore_can_get_now= 534,588,204 memstore_alloc_pos= 2,524,971,008 memstore_frozen_pos= 2,466,250,752 memstore_reclaimed_pos= 2,468,347,904 [TENANT_MEMORY] tenant_id= 1,004 now= 1,742,176,618,991,305 active_memstore_used= 272,629,760 total_memstore_used= 272,629,760 total_memstore_hold= 272,629,760 memstore_freeze_trigger_limit= 1,546,188,220 memstore_limit= 7,730,941,100 mem_tenant_limit= 15,461,882,266 mem_tenant_hold= 7,391,244,288 max_mem_memstore_can_get_now= 11,293,960,602 memstore_alloc_pos= 11,207,180,288 memstore_frozen_pos= 10,934,550,528 memstore_reclaimed_pos= 10,934,550,528 [2025-03-17 09:56:58.992076] INFO [STORAGE] print_tenant_usage (ob_tenant_memory_printer.cpp:114) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=10] [CHUNK_MGR] limit= 60,414,278,860 hold= 16,041,082,880 total_hold= 16,320,036,864 used= 15,275,225,088 freelists_hold= 765,857,792 total_maps= 97,965 total_unmaps= 90,814 large_maps= 164 large_unmaps= 0 huge_maps= 55 huge_unmaps= 46 resident_size= 0 unmanaged_memory_size= 188,051,456 virtual_memory_used= 17,468,538,880 [CHUNK_MGR] 2 MB_CACHE: hold= 538,968,064 count= 257 pushes= 682,144,743 pops= 682,144,486 maps= 97,746 unmaps= 90,768 [CHUNK_MGR] 4 MB_CACHE: hold= 55,107,584 count= 26 pushes= 443,881 pops= 443,855 maps= 78 unmaps= 0 [CHUNK_MGR] 6 MB_CACHE: hold= 27,918,336 count= 6 pushes= 99,218 pops= 99,212 maps= 30 unmaps= 0 [CHUNK_MGR] 8 MB_CACHE: hold= 25,247,744 count= 4 pushes= 1,570,636 pops= 1,570,632 maps= 7 unmaps= 0 [CHUNK_MGR] 10 MB_CACHE: hold= 101,818,368 count= 11 pushes= 556 pops= 545 maps= 28 unmaps= 0 [CHUNK_MGR] 12 MB_CACHE: hold= 0 count= 0 pushes= 4 pops= 4 maps= 4 unmaps= 0 [CHUNK_MGR] 14 MB_CACHE: hold= 0 count= 0 pushes= 0 pops= 0 maps= 2 unmaps= 0 [CHUNK_MGR] 16 MB_CACHE: hold= 0 count= 0 pushes= 2 pops= 2 maps= 4 unmaps= 0 [CHUNK_MGR] 18 MB_CACHE: hold= 16,797,696 count= 1 pushes= 15 pops= 14 maps= 10 unmaps= 0 [CHUNK_MGR] 20 MB_CACHE: hold= 0 count= 0 pushes= 0 pops= 0 maps= 1 unmaps= 0 [2025-03-17 09:56:58.992100] INFO print (ob_malloc_time_monitor.cpp:37) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=7] [MALLOC_TIME_MONITOR] show the distribution of ob_malloc's cost_time [MALLOC_TIME_MONITOR] [ 1000, 10000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 10000, 100000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 100000, 1000000): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [MALLOC_TIME_MONITOR] [ 1000000, 9223372036854775807): delta_total_cost_time= 0, delta_count= 0, avg_cost_time= 0 [2025-03-17 09:56:58.993501] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA584-0-0] [lt=61][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:58.994979] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:56:58.995505] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=38] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40319798476, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.011356] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=20] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:59.011431] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=10] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:59.025018] INFO [CLOG] do_thread_task_ (ob_remote_fetch_log_worker.cpp:250) [117800][T1_RFLWorker][T1][YB420A0B093E-00062FB7EE254125-0-0] [lt=24] ObRemoteFetchWorker is running(thread_index=0) [2025-03-17 09:56:59.029183] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=17] Slow write(duration=65244, write_len=8192) [2025-03-17 09:56:59.029224] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=39] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107781, offset=57286500}, end_cursor:ObLogCursor{file_id=5, log_id=1107782, offset=57286806}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b1770c050, len:4252, log_data_len:306, seq:1107781, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:59.029430] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731A-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:56:59.029469] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731A-0-0] [lt=24] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176618655657002, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176618655657002, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:59.029495] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:197) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=12] skip repeat study(ret=0, this={this:0x7f7b0abe7030, last_study_timestamp:1742176200088049, last_refresh_timestamp:1742176559026848, last_reclaim_timestamp:1742176200091277, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742175963158354000, v:0}, is_inited:true}, current_timestamp=1742176619029493, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:56:59.030600] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=29] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176199999845000, snapshot_type=1, create_time=1742176200000700, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:56:59.030630] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=29] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176443190492000, snapshot_type=1, create_time=1742176443202093, addr="192.168.1.63:2882", gc_status=0) [2025-03-17 09:56:59.030637] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176084587580000, snapshot_type=1, create_time=1742176084599019, addr="192.168.1.64:2882", gc_status=0) [2025-03-17 09:56:59.030643] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176563851152000, snapshot_type=1, create_time=1742176563862288, addr="192.168.1.65:2882", gc_status=0) [2025-03-17 09:56:59.030648] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176199632131000, snapshot_type=2, create_time=1742176200000700, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:56:59.030653] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176442823986000, snapshot_type=2, create_time=1742176443202093, addr="192.168.1.63:2882", gc_status=0) [2025-03-17 09:56:59.030659] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176084351075000, snapshot_type=2, create_time=1742176084599019, addr="192.168.1.64:2882", gc_status=0) [2025-03-17 09:56:59.030664] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176563508221000, snapshot_type=2, create_time=1742176563862288, addr="192.168.1.65:2882", gc_status=0) [2025-03-17 09:56:59.030676] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=11] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176199831990000, snapshot_type=3, create_time=1742176200000700, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:56:59.030684] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176442962658000, snapshot_type=3, create_time=1742176443202093, addr="192.168.1.63:2882", gc_status=0) [2025-03-17 09:56:59.030691] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=8] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176084111238000, snapshot_type=3, create_time=1742176084599019, addr="192.168.1.64:2882", gc_status=0) [2025-03-17 09:56:59.030699] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=1742176563851152000, snapshot_type=3, create_time=1742176563862288, addr="192.168.1.65:2882", gc_status=0) [2025-03-17 09:56:59.030704] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=5] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176200000700, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:56:59.030711] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176443202093, addr="192.168.1.63:2882", gc_status=0) [2025-03-17 09:56:59.030719] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176084599019, addr="192.168.1.64:2882", gc_status=0) [2025-03-17 09:56:59.030726] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176563862288, addr="192.168.1.65:2882", gc_status=0) [2025-03-17 09:56:59.030870] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:620) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=7] succeed to update global reserved snapshot(ret=0, this={this:0x7f7b0abe7030, last_study_timestamp:1742176200088049, last_refresh_timestamp:1742176559026848, last_reclaim_timestamp:1742176200091277, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742175963158354000, v:0}, is_inited:true}, global_reserved_snapshot_={val:1742175963158354000, v:0}, reserved_snapshot={val:1742176084111238000, v:0}) [2025-03-17 09:56:59.030895] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:554) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=25] multi-version garbage collector refresh successfully(this={this:0x7f7b0abe7030, last_study_timestamp:1742176200088049, last_refresh_timestamp:1742176559026848, last_reclaim_timestamp:1742176200091277, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176084111238000, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1742176084111238000, v:0}, reserved_snapshot_type:3, reserved_status:-1, reserved_create_time:1742176084599019, reserved_addr:"192.168.1.64:2882", is_this_server_disabled:false, status:0}) [2025-03-17 09:56:59.030912] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:232) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=16] repeat refresh successfully(ret=0, this={this:0x7f7b0abe7030, last_study_timestamp:1742176200088049, last_refresh_timestamp:1742176619030911, last_reclaim_timestamp:1742176200091277, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176084111238000, v:0}, is_inited:true}, current_timestamp=1742176619029525, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2025-03-17 09:56:59.030923] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:267) [117832][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=10] skip repeat reclaim(ret=0, this={this:0x7f7b0abe7030, last_study_timestamp:1742176200088049, last_refresh_timestamp:1742176619030911, last_reclaim_timestamp:1742176200091277, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176084111238000, v:0}, is_inited:true}, current_timestamp=1742176619030922, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:56:59.031876] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731A-0-0] [lt=23][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:59.052187] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=8] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=3) [2025-03-17 09:56:59.055297] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [3539421][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=56] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f79a77e82b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:56:59.058747] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:56:59.060873] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859691409}], scn_array=[cnt:1, {val:1742176618994499000, v:0}], curr_size=588, accum_size=4359, time_cost=65880) [2025-03-17 09:56:59.062762] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [117559][Blacklist][T0][Y0-0000000000000000-0-0] [lt=27] blacklist_loop exec finished(cost_time=30, is_enabled=true, send_cnt=0) [2025-03-17 09:56:59.063038] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=29] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:59.063085] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=24] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:59.077412] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859691997}], scn_array=[cnt:1, {val:1742176619031758000, v:0}], curr_size=1132, accum_size=5491, time_cost=16433) [2025-03-17 09:56:59.080162] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.086604] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107782, offset=57286806}, end_cursor:ObLogCursor{file_id=5, log_id=1107783, offset=57287103}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17710050, len:3946, log_data_len:297, seq:1107782, flush_finish:false, flush_ret:0}) [2025-03-17 09:56:59.086767] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731A-0-0] [lt=27] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:56:59.086807] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731A-0-0] [lt=26] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176618655657003, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176618655657003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:56:59.118463] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=30] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:56:59.122089] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:56:59.122140] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=27] [IO STATUS GROUP](tenant_id=1, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:20480, iops:2, bw:40897, limit:9223372036854775807, [delay/us]:prepare:11, schedule:8, submit:1, rt:5603, total:5624) [2025-03-17 09:56:59.122248] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=100] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:SHARED_BLOCK_ReaderWriter_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8179, [delay/us]:prepare:1, schedule:21, submit:1, rt:11100, total:11125) [2025-03-17 09:56:59.122263] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:SHARED_MACRO_BLOCK_MGR_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:20480, iops:2, bw:40897, [delay/us]:prepare:1, schedule:17, submit:0, rt:1751, total:1770) [2025-03-17 09:56:59.122274] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32713, [delay/us]:prepare:1, schedule:26, submit:1, rt:2339, total:2368, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:59.122289] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4437, iops:12, bw:53167, [delay/us]:prepare:5, schedule:19, submit:1450, rt:15199, total:16675) [2025-03-17 09:56:59.122297] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=2, ops=5, iops=7, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:59.122333] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=32] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:6827, avg_iops:12, avg_bw:81785, [delay/us]: prepare:1, schedule:24, submit:1, device:3701, total:3728) [2025-03-17 09:56:59.122338] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:20480, avg_iops:2, avg_bw:40892, [delay/us]: prepare:11, schedule:8, submit:1, device:5603, total:5624) [2025-03-17 09:56:59.122350] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4437, avg_iops:12, avg_bw:53160, [delay/us]: prepare:5, schedule:19, submit:1450, device:15199, total:16675) [2025-03-17 09:56:59.122363] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:56:59.122396] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32713, [delay/us]:prepare:1, schedule:27, submit:1, rt:4996, total:5026, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:56:59.122413] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4608, iops:16, bw:73606, [delay/us]:prepare:5, schedule:19, submit:29, rt:27932, total:27987) [2025-03-17 09:56:59.122432] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS TENANT](tenant_id=1003, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:59.122461] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=27] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32713, [delay/us]: prepare:1, schedule:27, submit:1, device:4996, total:5026) [2025-03-17 09:56:59.122476] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4608, avg_iops:16, avg_bw:73604, [delay/us]: prepare:5, schedule:19, submit:29, device:27932, total:27987) [2025-03-17 09:56:59.122493] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:56:59.122513] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:6144, iops:2, bw:12267, [delay/us]:prepare:6, schedule:29, submit:1, rt:36999, total:37036) [2025-03-17 09:56:59.122540] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4300, iops:20, bw:85872, [delay/us]:prepare:5, schedule:26, submit:4839, rt:11092, total:15964) [2025-03-17 09:56:59.122548] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:59.122574] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:6144, avg_iops:2, avg_bw:12267, [delay/us]: prepare:6, schedule:29, submit:1, device:36999, total:37036) [2025-03-17 09:56:59.122587] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4301, avg_iops:20, avg_bw:85870, [delay/us]: prepare:5, schedule:26, submit:4839, device:11092, total:15964) [2025-03-17 09:56:59.122599] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:56:59.122624] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4089, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:28988, total:28995) [2025-03-17 09:56:59.122646] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:56:59.122671] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4089, [delay/us]: prepare:0, schedule:0, submit:600000000, device:28988, total:28995) [2025-03-17 09:56:59.122689] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:56:59.122704] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:59.122713] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122721] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122729] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122737] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122744] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122752] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122760] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122769] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:56:59.122777] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122788] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122796] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122805] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:56:59.122815] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS SERVER] net_in=182kB/s, net_out=108kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:56:59.128972] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=37] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=12478464) [2025-03-17 09:56:59.129063] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=34] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:59.132187] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1742176619131906000, start_id=1742176619131906000, end_id=1742176619131906001) [2025-03-17 09:56:59.132297] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=34] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=8248) [2025-03-17 09:56:59.144883] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=10] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1742176618945509000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=199368, in_cluster_service=true, cluster_version={val:1742176618746046000, v:0}, min_cluster_version={val:1742176618746046000, v:0}, max_cluster_version={val:1742176618746046000, v:0}, get_cluster_version_err=0, cluster_version_delta=398831, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=8524936, succ_cluster_heartbeat_count=14738, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176199632131000, v:0}, local_cluster_delta=419512746, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:56:59.150852] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=19] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:56:59.150888] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=29] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:56:59.150900] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=10] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2025-03-17 09:56:59.150907] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=5] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=2, times=11756) [2025-03-17 09:56:59.156183] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=24] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:56:59.156238] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=36] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1742176619156220], gts:1742176619155215000, latest_srr:[mts=1742176619156220]}) [2025-03-17 09:56:59.156518] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1742176619156497], gts:1742176619156275000, latest_srr:[mts=1742176619156497]}) [2025-03-17 09:56:59.158443] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:400) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E816-0-0] [lt=33] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2025-03-17 09:56:59.164919] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463510221}], scn_array=[cnt:1, {val:1742176619145671000, v:0}], curr_size=586, accum_size=1295, time_cost=18359) [2025-03-17 09:56:59.174526] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=21] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:56:59.179811] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872702347}], scn_array=[cnt:1, {val:1742176619156275000, v:0}], curr_size=122, accum_size=1098, time_cost=20998) [2025-03-17 09:56:59.179921] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414492}], scn_array=[cnt:1, {val:1742176619156275000, v:0}], curr_size=122, accum_size=1220, time_cost=21085) [2025-03-17 09:56:59.180226] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.181158] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539188][T1004_Occam][T1004][YB420A0B093E-00062FB86F654125-0-0] [lt=17] ====== tenant freeze timer task ====== [2025-03-17 09:56:59.181705] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=27] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:56:59.191166] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=22] PNIO [ratelimit] time: 1742176619191164, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007618, add_bytes: 0 [2025-03-17 09:56:59.195902] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=35] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40311409868, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.196174] INFO [BALANCE.TRANSFER] do_work (ob_tenant_transfer_service.cpp:146) [3539200][T1004_TntTransf][T1004][YB420A0B093E-00062FB7FC4623F0-0-0] [lt=33] [TENANT_TRANSFER] finish one round(ret=0, ret="OB_SUCCESS", all_tasks_count=0, thread_task_count=0, thread_idx=0) [2025-03-17 09:56:59.215347] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539374][T1003_Occam][T1003][YB420A0B093E-00062FB804154126-0-0] [lt=21] ====== tenant freeze timer task ====== [2025-03-17 09:56:59.227492] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=50][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859694466}], scn_array=[cnt:1, {val:1742176619194715000, v:0}], curr_size=588, accum_size=7416, time_cost=32475) [2025-03-17 09:56:59.246935] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:59.246975] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:56:59.247405] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176617654369002, replayable_scn:1742176617654369002, readable_scn:1742176617654369001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176619031758000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:59.247446] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176617654369002, replayable_scn:1742176617654369002, readable_scn:1742176617654369001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176619031758000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:59.247456] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176617654369002, replayable_scn:1742176617654369002, readable_scn:1742176617654369001, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176619031758000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:56:59.248348] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFA9-0-0] [lt=5][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:59.248391] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:56:59.248402] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:56:59.252318] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=2) [2025-03-17 09:56:59.265125] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=18] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7068, ref_cnt_:1, raw_buf_:0x7f7ac25a2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78623-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176619264737, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:59.265205] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=77][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.265228] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=22][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.265250] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=22][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.265261] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7068, ref_cnt_:1, raw_buf_:0x7f7ac25a2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78623-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176619264737, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:59.265302] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=34][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7068, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78623-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176619264737, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:350, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176619265297}}, part_id:-1}) [2025-03-17 09:56:59.265311] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=5][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:59.265673] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=30][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.265713] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=39][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:59.265720] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a41039510, src_buf_size=295) [2025-03-17 09:56:59.265727] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6528, ref_cnt_:1, raw_buf_:0x7f7a41038150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78623-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176619265358, enqueue_used:0, dequeue_used:41, submit_used:0, return_used:253, callback_enqueue_used:2, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:59.265776] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6528, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78623-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f34e080, time_log:{begin_ts:1742176619265358, enqueue_used:0, dequeue_used:41, submit_used:0, return_used:253, callback_enqueue_used:2, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176619265762}}, part_id:-1}) [2025-03-17 09:56:59.265832] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78623-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade972d80, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222520047, assoc_session_id:3222520047, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176619263733, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176619263732, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a393d21f8, id:3222520047, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade972d80}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.274141] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13873, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:56:59.274193] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=51][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414614}], scn_array=[cnt:1, {val:1742176619156275001, v:0}], curr_size=122, accum_size=1342, time_cost=15222) [2025-03-17 09:56:59.274219] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1342) [2025-03-17 09:56:59.274235] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=32, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:56:59.274258] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1342) [2025-03-17 09:56:59.274300] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=8, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:56:59.274314] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=13977, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:56:59.274329] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=38, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:56:59.280296] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.284258] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=2010, avg_log_gen_to_submit_time=2023, avg_log_submit_to_flush_time=14850, avg_log_submit_to_slide_time=14866) [2025-03-17 09:56:59.284342] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=4747, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:59.293198] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=22] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:56:59.293916] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=25] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:56:59.293975] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=19] inner load succ(latest_freeze_info={frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176617637226000, v:0}, frozen_statuses_.count():6, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741314277005166000, v:0}, schema_version:1741314275575456, data_version:17180067072}, {frozen_version:{val:1741370403684882000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741456800355942000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741543202134877000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}]}) [2025-03-17 09:56:59.306388] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [3539406][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=17] LogLoopThread round_cost_time(us)(round_cost_time=13) [2025-03-17 09:56:59.306894] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=33] PNIO [ratelimit] time: 1742176619306892, bytes: 2535005321643, bw: 0.166765 MB/s, add_ts: 1007627, add_bytes: 176200 [2025-03-17 09:56:59.309489] INFO [LIB] stat (utility.h:1210) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:56:59.316266] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=22] ====== traversal_flush timer task ====== [2025-03-17 09:56:59.316394] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539394][T1003_TimerWK2][T1003][YB420A0B093E-00062FB7F9CE9B35-0-0] [lt=30] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742157237104149001, v:0}}, }, max_consequent_callbacked_scn={val:1742176619194715000, v:0}, *this={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176626190689000, latest_log_ts:{val:1742176609587544001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:59.316484] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [3539394][T1003_TimerWK2][T1003][YB420A0B093E-00062FB7F9CE9B35-0-0] [lt=86] succeed to traversal_flush(ret=0, ls_cnt=1) [2025-03-17 09:56:59.319225] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DB-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13701][2nd=13702]}, offset=1979029) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.322189] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A79-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:56:59.322215] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A79-0-0] [lt=25] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:56:59.342481] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=56][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859695176}], scn_array=[cnt:1, {val:1742176619331679000, v:0}], curr_size=587, accum_size=8125, time_cost=10141) [2025-03-17 09:56:59.343836] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=12] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:59.345402] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=3][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.345794] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222519615, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2964a0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:59.345835] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634328} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:59.345866] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=27] end stmt(ret=0, tx_id=9634328, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2964a0, tx_id:{txid:9634328}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519615, assoc_session_id:3222519615, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176619344409, active_ts:1742176619344409, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649344403, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463459176247, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619341237}], exec_info_reap_ts:1742176619341236, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619341233, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222519615, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2964a0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.345977] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634328}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176649344403, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619344409}{self_ls_id:{id:1}, session_id:3222519615, associated_session_id:3222519615, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634328}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463459176247, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619344409, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634328} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.346059] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=72] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634328}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176649344403, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619344409}{self_ls_id:{id:1}, session_id:3222519615, associated_session_id:3222519615, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634328}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463459176247, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619344409, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634328} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.346115] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=52] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634328}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176619345973, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:59.346138] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=21] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2964a0, tx_id:{txid:9634328}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519615, assoc_session_id:3222519615, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176619344409, active_ts:1742176619344409, commit_ts:-1, finish_ts:1742176619345469, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649344403, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463459176247, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619341237}], exec_info_reap_ts:1742176619341236, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619341233, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:59.346221] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=65] get number(ret=0, service_type_=0, range=1, base_id=1742176619345469000, start_id=1742176619345469000, end_id=1742176619345469001) [2025-03-17 09:56:59.346250] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FD-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:59.346289] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=4089, avg_group_log_size=371) [2025-03-17 09:56:59.347810] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=29] gc stale ls task succ [2025-03-17 09:56:59.357649] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=33] ====== check clog disk timer task ====== [2025-03-17 09:56:59.357680] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=26] get_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:56:59.380368] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.381014] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=54][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872702591}], scn_array=[cnt:1, {val:1742176619156275002, v:0}], curr_size=122, accum_size=1342, time_cost=22096) [2025-03-17 09:56:59.381050] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1342) [2025-03-17 09:56:59.381098] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1342) [2025-03-17 09:56:59.381164] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:59.383478] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175476][T1_L0_G5][T1][YB420A0B0941-0006302366DD6B5C-0-0] [lt=32] get number(ret=0, service_type_=0, range=1, base_id=1742176619382688000, start_id=1742176619382688000, end_id=1742176619382688001) [2025-03-17 09:56:59.396267] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=49] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40305118412, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.415692] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416014] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416073] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416077] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416455] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=51] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416708] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.416720] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.430597] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859695763}], scn_array=[cnt:1, {val:1742176619394567000, v:0}], curr_size=588, accum_size=8713, time_cost=35271) [2025-03-17 09:56:59.430646] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=48] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8713) [2025-03-17 09:56:59.430665] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=24, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:56:59.430714] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=8, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:56:59.430728] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=14] [PALF STAT LOG LIFE TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=24959, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:56:59.430849] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=120] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=16, avg_log_gen_to_freeze_time=2658, avg_log_gen_to_submit_time=2670, avg_log_submit_to_flush_time=22269, avg_log_submit_to_slide_time=22288) [2025-03-17 09:56:59.430872] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=48, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:56:59.430885] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8713, hit_cnt=16, read_cnt=16, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:59.432638] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=26] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:59.447206] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=11] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:59.449009] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7848, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FE-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176619448560, enqueue_used:2, dequeue_used:33, submit_used:1, return_used:376, callback_enqueue_used:3, callback_dequeue_used:32, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:56:59.449063] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.449441] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222518240, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3fc700}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:59.449483] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=29] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2149}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634332} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:59.449513] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=26] end stmt(ret=0, tx_id=9634332, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3fc700, tx_id:{txid:9634332}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518240, assoc_session_id:3222518240, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176619447968, active_ts:1742176619447968, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649447961, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463562713858, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619446892}], exec_info_reap_ts:1742176619446891, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619446888, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222518240, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3fc700}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.449624] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634332}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176649447961, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619447968}{self_ls_id:{id:1}, session_id:3222518240, associated_session_id:3222518240, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634332}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463562713858, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619447968, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634332} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.449691] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=56] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634332}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176649447961, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619447968}{self_ls_id:{id:1}, session_id:3222518240, associated_session_id:3222518240, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634332}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463562713858, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619447968, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634332} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.449729] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=34] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634332}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176619449620, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:59.449745] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3fc700, tx_id:{txid:9634332}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518240, assoc_session_id:3222518240, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176619447968, active_ts:1742176619447968, commit_ts:-1, finish_ts:1742176619449036, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649447961, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463562713858, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619446892}], exec_info_reap_ts:1742176619446891, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619446888, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:59.449817] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FE-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:59.452457] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=32] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=1) [2025-03-17 09:56:59.465707] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=33] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}]) [2025-03-17 09:56:59.480442] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=58][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.480996] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414858}], scn_array=[cnt:1, {val:1742176619156275003, v:0}], curr_size=122, accum_size=244, time_cost=21984) [2025-03-17 09:56:59.481477] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872702713}], scn_array=[cnt:1, {val:1742176619156275003, v:0}], curr_size=122, accum_size=122, time_cost=22482) [2025-03-17 09:56:59.511506] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:56:59.511589] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:56:59.516926] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=23][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.516976] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=48][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.516999] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=22][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.517013] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=13][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6060, ref_cnt_:1, raw_buf_:0x7f7a3f2b2150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0073-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176619516370, enqueue_used:2, dequeue_used:20, submit_used:1, return_used:512, callback_enqueue_used:3, callback_dequeue_used:15, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:59.517065] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6060, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0073-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176619516370, enqueue_used:2, dequeue_used:20, submit_used:1, return_used:512, callback_enqueue_used:3, callback_dequeue_used:15, callback_finish_used:-1, end_used:1742176619517048}}, part_id:-1}) [2025-03-17 09:56:59.517072] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=29][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:59.517534] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=24][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.517569] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=35][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:59.517577] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaacbf510, src_buf_size=295) [2025-03-17 09:56:59.517585] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8100, ref_cnt_:1, raw_buf_:0x7f7aaacbe150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0073-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176619517156, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:329, callback_enqueue_used:1, callback_dequeue_used:20, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:59.517624] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8100, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0073-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176619517156, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:329, callback_enqueue_used:1, callback_dequeue_used:20, callback_finish_used:-1, end_used:1742176619517620}}, part_id:-1}) [2025-03-17 09:56:59.517704] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0073-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8352a0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222517287, assoc_session_id:3222517287, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176619516161, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176619508747, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222517287, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace8352a0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.523036] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.523099] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=61][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.523125] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=25][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.523136] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7a88, ref_cnt_:1, raw_buf_:0x7f7a3f34e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176619522537, enqueue_used:2, dequeue_used:25, submit_used:1, return_used:453, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:56:59.523199] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=51][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7a88, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a3f2b2080, time_log:{begin_ts:1742176619522537, enqueue_used:2, dequeue_used:25, submit_used:1, return_used:453, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176619523190}}, part_id:-1}) [2025-03-17 09:56:59.523222] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=19][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:59.523557] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=21][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:59.523649] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1A-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6bcbe00, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222512886, assoc_session_id:3222512886, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176619521478, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176619520381, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ad1c721f8, id:3222512886, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6bcbe00}, exec_ctx.get_errcode()=0) [2025-03-17 09:56:59.532891] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] submit log success(service_type_=0, last_id=1742176636190689000, limited_id=1742176636190689000) [2025-03-17 09:56:59.532927] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=35] submit log with lock success(service_type_=0, last_id=1742176636190689000, limited_id=1742176636190689000) [2025-03-17 09:56:59.534339] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=19] LogLoopThread round_cost_time(us)(round_cost_time=17) [2025-03-17 09:56:59.538990] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5712-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=188, pos=160) [2025-03-17 09:56:59.539615] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5712-0-0] [lt=33][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=28, pos=15) [2025-03-17 09:56:59.540381] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5712-0-0] [lt=17][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=178, pos=151) [2025-03-17 09:56:59.540812] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [3539416][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=35] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:56:59.558006] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT WRITE LOG COST TIME](cur_stat_count=15, stat_interval=1000000, avg (count/cost)=22982, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:56:59.558050] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859696351}], scn_array=[cnt:1, {val:1742176619394567001, v:0}], curr_size=122, accum_size=122, time_cost=48352) [2025-03-17 09:56:59.558113] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8125) [2025-03-17 09:56:59.559167] INFO [COMMON] try_expand_thread_count (ob_dynamic_thread_pool.cpp:473) [117637][][T0][Y0-0000000000000000-0-0] [lt=29] expand thread count(this=0x7f7b319af170, max_thread_cnt=128, cur_thread_count=2, inc_cnt=1, queue_size=2) [2025-03-17 09:56:59.559380] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [357893][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=0] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:3, threads_idle_time:1312801609526, tenant_id:500}, cur_thread_count=3, cnt=1, new_thread_count=4) [2025-03-17 09:56:59.559565] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [357893][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=36] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:3, threads_idle_time:1312801609526, tenant_id:500}, cur_thread_count=3, cnt=1, new_thread_count=4) [2025-03-17 09:56:59.563200] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:172) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=27] Eliminate task evict sql audit(request_manager_->get_tenant_id()=1, queue_size=589824, config_mem_limit_=193273528, request_manager_->get_size_used()=65536, evict_high_size_level=530841, evict_low_size_level=471859, allocator->allocated()=124784640, evict_high_mem_level=154618822, evict_low_mem_level=115964116) [2025-03-17 09:56:59.563239] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=37] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:56:59.563275] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=13] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:56:59.572932] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118852][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC97-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:10008, add_timestamp:1742176619572923}) [2025-03-17 09:56:59.572957] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3186-0-0] [lt=21] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=10008, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.573006] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118852][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC97-0-0] [lt=25] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12358, add_timestamp:1742176619573003}) [2025-03-17 09:56:59.573024] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118852][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC97-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12396, add_timestamp:1742176619573023}) [2025-03-17 09:56:59.573284] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3186-0-0] [lt=48] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=10008, locations=[{table_id:10008, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.573336] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3186-0-0] [lt=51] success to process renew task(task={tenant_id:1, table_id:10008, add_timestamp:1742176619572923}, locations=[{table_id:10008, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}, {table_id:10008, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573263, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.573409] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3187-0-0] [lt=59] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12358, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573403, sql_renew_time:1742176619573403, is_mark_fail:false}]) [2025-03-17 09:56:59.573438] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3187-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:12358, add_timestamp:1742176619573003}, locations=[{table_id:12358, partition_id:12358, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573403, sql_renew_time:1742176619573403, is_mark_fail:false}]) [2025-03-17 09:56:59.573468] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3188-0-0] [lt=19] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12396, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573463, sql_renew_time:1742176619573463, is_mark_fail:false}]) [2025-03-17 09:56:59.573488] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3188-0-0] [lt=19] success to process renew task(task={tenant_id:1, table_id:12396, add_timestamp:1742176619573023}, locations=[{table_id:12396, partition_id:12396, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619573463, sql_renew_time:1742176619573463, is_mark_fail:false}]) [2025-03-17 09:56:59.576157] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118852][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC97-0-0] [lt=7][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:59.580258] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463511638}], scn_array=[cnt:1, {val:1742176619546981000, v:0}], curr_size=587, accum_size=2713, time_cost=33207) [2025-03-17 09:56:59.580293] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859696473}], scn_array=[cnt:1, {val:1742176619532124001, v:0}], curr_size=660, accum_size=782, time_cost=22071) [2025-03-17 09:56:59.580417] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:846) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=5] update id meta(ret=0, service_type=0, limited_id=1742176636190689000, latest_scn={val:1742176619532124001, v:0}, *this={tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}) [2025-03-17 09:56:59.580490] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=72] handle submit callback(service_type_=0, success=true, used_ts=47600, log_ts={val:1742176619532124001, v:0}, limited_id=1742176636190689000) [2025-03-17 09:56:59.580502] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:564) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2025-03-17 09:56:59.580512] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:602) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=10] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2025-03-17 09:56:59.580521] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.580534] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=12] get number(ret=0, service_type_=0, range=1, base_id=1742176619580015000, start_id=1742176619580015000, end_id=1742176619580015001) [2025-03-17 09:56:59.583867] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162414980}], scn_array=[cnt:1, {val:1742176619156275004, v:0}], curr_size=122, accum_size=366, time_cost=24743) [2025-03-17 09:56:59.583922] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872702835}], scn_array=[cnt:1, {val:1742176619156275004, v:0}], curr_size=122, accum_size=244, time_cost=24818) [2025-03-17 09:56:59.586322] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=22][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:449, tid:117786}]) [2025-03-17 09:56:59.596615] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=27] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40300924108, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.600365] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:59.602184] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=0][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.602238] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=52][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.602264] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=25][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.602276] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7458, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FF-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619601752, enqueue_used:2, dequeue_used:31, submit_used:2, return_used:370, callback_enqueue_used:3, callback_dequeue_used:20, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.602337] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=48][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7458, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FF-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619601752, enqueue_used:2, dequeue_used:31, submit_used:2, return_used:370, callback_enqueue_used:3, callback_dequeue_used:20, callback_finish_used:-1, end_used:1742176619602328}}, part_id:-1}) [2025-03-17 09:56:59.602388] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=43][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619601752, enqueue_used:2, dequeue_used:31, submit_used:2, return_used:370, callback_enqueue_used:3, callback_dequeue_used:20, callback_finish_used:197, end_used:-49}}) [2025-03-17 09:56:59.602413] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=24][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.602434] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=18][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:59.602761] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=39][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.602793] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=32][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:59.602800] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d1c315a, src_buf_size=1611) [2025-03-17 09:56:59.602808] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8bb0, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FF-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619602446, enqueue_used:1, dequeue_used:22, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.602850] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=34][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8bb0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF633FF-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619602446, enqueue_used:1, dequeue_used:22, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:25, callback_finish_used:-1, end_used:1742176619602844}}, part_id:-1}) [2025-03-17 09:56:59.602881] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619602446, enqueue_used:1, dequeue_used:22, submit_used:1, return_used:264, callback_enqueue_used:1, callback_dequeue_used:25, callback_finish_used:154, end_used:-70}}) [2025-03-17 09:56:59.602955] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=73][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.602988] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=31][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.603019] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=31][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.603050] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=27][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:59.603081] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=30][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.603107] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=25][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.603181] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=73][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:59.603187] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=5][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:59.603193] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=5][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:59.603203] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=5][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:59.603210] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649501607, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:59.603313] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=102][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649501607, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a91c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 191A8C1A80300600, int: 1742176619600409}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}}) [2025-03-17 09:56:59.603398] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=82][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a91c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 191A8C1A80300600, int: 1742176619600409}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:59.603460] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=60][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:59.603471] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:59.603482] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}, ret=-4016) [2025-03-17 09:56:59.603549] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=13][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7aa426ae50, local_datums:0x7f7aa426aaa0}, datums:[col_id=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a91c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a92d, hex: 191A8C1A80300600, int: 1742176619600409}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a935, hex: 191A8C1A80300600, int: 1742176619600409}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a93d, hex: 191A8C1A80300600, int: 1742176619600409}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a945, hex: 0A00000000000000, int: 10}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a94d, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7aa426a955, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a95b, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 36, flag: 0, null: 0, ptr: 0x7f7aa426a96c, hex: 44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829, cstr: DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a990, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a999, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a9a1, hex: 00C40FE7FF2F0600, int: 1741626000000000}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a9a9, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a9ba, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176649501607, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176649501607, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176619595002000, v:0}, tx_id:{txid:9634335}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634335}, tx_desc:{this:0x7f7acf376e40, tx_id:{txid:9634335}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519993, assoc_session_id:3222519993, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176619600385, active_ts:1742176619600385, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649600378, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463715844864, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176619595005}], exec_info_reap_ts:1742176619595004, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619595001, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:59.603684] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=133][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:59.603697] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a91c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a92d, hex: 191A8C1A80300600, int: 1742176619600409}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a91c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a92d, hex: 191A8C1A80300600, int: 1742176619600409}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.600409"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649501607, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:59.603807] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=109][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:59.603815] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=7][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:59.603831] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=7][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:59.603840] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=8][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:59.603862] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=17][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:59.603873] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:59.603884] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:59.603896] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=11][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:59.603908] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=11][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176649601607, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388533, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf376e40, tx_id:{txid:9634335}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519993, assoc_session_id:3222519993, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176619600385, active_ts:1742176619600385, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649600378, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463715844864, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176619595005}], exec_info_reap_ts:1742176619595004, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619595001, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634335}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649501607, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649501607, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649501607, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:59.604932] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=1022][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:59.604945] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=11][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:59.604959] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=12][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:59.604966] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=6][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:59.604972] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=6][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:59.604980] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=6][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:59.604996] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=14][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:59.605005] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=8][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:59.605014] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:59.605023] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:59.605032] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:59.605042] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:59.605052] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}) [2025-03-17 09:56:59.605066] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=13][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:59.605081] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=14][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:59.605107] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=15] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222519993, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf376e40}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:59.605166] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=23] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634335} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:59.605199] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=28] end stmt(ret=0, tx_id=9634335, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf376e40, tx_id:{txid:9634335}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519993, assoc_session_id:3222519993, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176619600385, active_ts:1742176619600385, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649600378, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463715844864, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619595005}], exec_info_reap_ts:1742176619595004, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619595001, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222519993, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf376e40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.605252] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=51][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:59.605262] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:59.605271] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:59.605289] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:59.605301] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=11][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:59.605313] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')) [2025-03-17 09:56:59.605340] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:59.605351] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), is_user_sql=false) [2025-03-17 09:56:59.605361] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=9][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619600409), usec_to_time(1742176619600409), 10, usec_to_time(1742176619600409), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), conn=0x7f7aeb4ab0e0) [2025-03-17 09:56:59.605376] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=14][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:59.605492] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=10][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634335}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176649600378, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619600385}{self_ls_id:{id:1}, session_id:3222519993, associated_session_id:3222519993, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634335}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463715844864, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619601479, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634335} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.605578] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=70] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634335}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176649600378, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619600385}{self_ls_id:{id:1}, session_id:3222519993, associated_session_id:3222519993, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634335}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463715844864, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619601479, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634335} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.605646] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=61] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634335}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176619605487, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:59.605667] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=19] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf376e40, tx_id:{txid:9634335}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222519993, assoc_session_id:3222519993, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176619600385, active_ts:1742176619600385, commit_ts:-1, finish_ts:1742176619604683, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176649600378, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463715844864, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619595005}], exec_info_reap_ts:1742176619595004, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619595001, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:59.605793] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=38][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:59.605805] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=12][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:59.605816] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF633FF-0-0] [lt=8][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:59.611227] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859697133}], scn_array=[cnt:1, {val:1742176619595002000, v:0}], curr_size=588, accum_size=1370, time_cost=15707) [2025-03-17 09:56:59.611476] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1742176619394567000, v:0}, new_max_version={val:1742176619394567000, v:0}, last_min_version={val:1742176619194715000, v:0}, last_max_version={val:1742176619194715000, v:0}, total_time_us=17275, delta_us=216907) [2025-03-17 09:56:59.611520] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=42] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1742176619394567000, v:0}, min_version={val:1742176619394567000, v:0}, max_version={val:1742176619394567000, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:56:59.613465] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=56] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:56:59.629168] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=17] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=29116416) [2025-03-17 09:56:59.629233] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=30] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=66, size_used=0, mem_used=0) [2025-03-17 09:56:59.629331] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=51] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:56:59.635775] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=39] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.635797] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636059] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636102] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=32] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636099] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636120] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636182] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636252] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=23] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636407] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.636464] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=9] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:56:59.641480] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=43] gc stale ls task succ [2025-03-17 09:56:59.652588] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=0) [2025-03-17 09:56:59.653607] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=40] ====== check clog disk timer task ====== [2025-03-17 09:56:59.653647] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=34] get_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:56:59.657166] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=19] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=363, get_gts_with_stc_cnt=4, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=2, try_wait_gts_elapse_cnt=0) [2025-03-17 09:56:59.659170] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:59.659208] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:59.659243] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:56:59.659259] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:56:59.674051] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=31] gc stale ls task succ [2025-03-17 09:56:59.680682] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=61][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.688245] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463512225}], scn_array=[cnt:1, {val:1742176619656135000, v:0}], curr_size=122, accum_size=2835, time_cost=10924) [2025-03-17 09:56:59.697084] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=35] current monitor number(seq_=-1) [2025-03-17 09:56:59.710509] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=9] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:56:59.711496] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=43] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.217825093269348145e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=608, Throttle Trigger(MB)=365, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.288100158762217688e+02) [2025-03-17 09:56:59.712382] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=64][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.712434] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=50][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.712455] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.712471] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=15][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c75c0, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63400-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619711907, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:420, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.712535] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=52][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c75c0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63400-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619711907, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:420, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:-1, end_used:1742176619712526}}, part_id:-1}) [2025-03-17 09:56:59.712577] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=46][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619711907, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:420, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:187, end_used:-40}}) [2025-03-17 09:56:59.712633] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=55][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.712655] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=18][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:59.712972] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=31][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.713001] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=29][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:59.713007] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a0e5bd15a, src_buf_size=1611) [2025-03-17 09:56:59.713012] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7f98, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63400-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619712666, enqueue_used:2, dequeue_used:29, submit_used:1, return_used:248, callback_enqueue_used:1, callback_dequeue_used:24, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.713040] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=22][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7f98, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63400-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619712666, enqueue_used:2, dequeue_used:29, submit_used:1, return_used:248, callback_enqueue_used:1, callback_dequeue_used:24, callback_finish_used:-1, end_used:1742176619713035}}, part_id:-1}) [2025-03-17 09:56:59.713064] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176619712666, enqueue_used:2, dequeue_used:29, submit_used:1, return_used:248, callback_enqueue_used:1, callback_dequeue_used:24, callback_finish_used:87, end_used:-23}}) [2025-03-17 09:56:59.713082] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=17][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.713091] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.713111] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=20][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.713128] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=15][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:59.713165] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=36][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.713195] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=29][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.713249] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=53][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:56:59.713257] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:59.713264] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:59.713275] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:59.713282] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649611769, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:59.713338] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=55][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649611769, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f79f02a098c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 59C88D1A80300600, int: 1742176619710553}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}}) [2025-03-17 09:56:59.713384] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=44][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f79f02a098c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 59C88D1A80300600, int: 1742176619710553}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:56:59.713417] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=32][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:59.713458] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=39][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:56:59.713466] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}, ret=-4016) [2025-03-17 09:56:59.713518] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=11][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f79f02a0ed0, local_datums:0x7f79f02a0b20}, datums:[col_id=0:{len: 27, flag: 0, null: 0, ptr: 0x7f79f02a098c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09a7, hex: 59C88D1A80300600, int: 1742176619710553}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09af, hex: 59C88D1A80300600, int: 1742176619710553}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09b7, hex: 59C88D1A80300600, int: 1742176619710553}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09bf, hex: 0900000000000000, int: 9}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09c7, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f79f02a09cf, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a09d5, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 49, flag: 0, null: 0, ptr: 0x7f79f02a09e6, hex: 44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029, cstr: DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f79f02a0a17, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a0a20, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a0a28, hex: 53B0C11DF92F0600, int: 1741596852858963}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f79f02a0a30, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f79f02a0a41, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176649611769, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176649611769, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176619656135000, v:0}, tx_id:{txid:9634336}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634336}, tx_desc:{this:0x7f7acf3f04c0, tx_id:{txid:9634336}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222506595, assoc_session_id:3222506595, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176619710352, active_ts:1742176619710352, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649710346, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463825942270, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176619709310}], exec_info_reap_ts:1742176619709309, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619709306, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:56:59.713628] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=108][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:56:59.713636] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f79f02a098c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09a7, hex: 59C88D1A80300600, int: 1742176619710553}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f79f02a098c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f79f02a09a7, hex: 59C88D1A80300600, int: 1742176619710553}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:56:59.710553"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176649611769, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:56:59.713731] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=94][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:56:59.713742] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=10][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:56:59.713757] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=11][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:59.713764] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:56:59.713788] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=19][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:56:59.713799] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=12][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:56:59.713813] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=13][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:59.713820] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:56:59.713828] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176649711769, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388543, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf3f04c0, tx_id:{txid:9634336}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222506595, assoc_session_id:3222506595, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176619710352, active_ts:1742176619710352, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649710346, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463825942270, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176619709310}], exec_info_reap_ts:1742176619709309, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619709306, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634336}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649611769, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649611769, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176649611769, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:570, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:56:59.714673] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=844][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:56:59.714679] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:59.714688] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:59.714692] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=4][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:56:59.714697] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=4][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:56:59.714701] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=4][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:56:59.714708] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:56:59.714715] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:56:59.714719] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=4][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:56:59.714726] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:56:59.714732] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:59.714740] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:56:59.714749] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}) [2025-03-17 09:56:59.714761] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=12][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:56:59.714774] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=12][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:56:59.714795] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=13] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222506595, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3f04c0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:56:59.714826] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634336} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:56:59.714850] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=20] end stmt(ret=0, tx_id=9634336, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3f04c0, tx_id:{txid:9634336}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222506595, assoc_session_id:3222506595, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176619710352, active_ts:1742176619710352, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649710346, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463825942270, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619709310}], exec_info_reap_ts:1742176619709309, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619709306, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222506595, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3f04c0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.714883] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=32][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:59.714890] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:56:59.714896] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:56:59.714907] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=6][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:56:59.714916] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, aret=-4016, ret=-4016) [2025-03-17 09:56:59.714924] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')) [2025-03-17 09:56:59.714941] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:56:59.714949] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC'), is_user_sql=false) [2025-03-17 09:56:59.714958] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=8][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176619710553), usec_to_time(1742176619710553), 9, usec_to_time(1742176619710553), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC'), conn=0x7f7ac70b3430) [2025-03-17 09:56:59.714965] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:56:59.715030] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=7][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634336}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176649710346, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619711397}{self_ls_id:{id:1}, session_id:3222506595, associated_session_id:3222506595, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634336}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463825942270, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619711397, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634336} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.715089] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=47] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634336}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176649710346, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176619711397}{self_ls_id:{id:1}, session_id:3222506595, associated_session_id:3222506595, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634336}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409463825942270, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176619711397, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634336} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:56:59.715132] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=33] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634336}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176619715026, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:56:59.715163] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=30] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3f04c0, tx_id:{txid:9634336}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222506595, assoc_session_id:3222506595, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176619710352, active_ts:1742176619710352, commit_ts:-1, finish_ts:1742176619714586, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176649710346, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409463825942270, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176619709310}], exec_info_reap_ts:1742176619709309, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176619709306, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:56:59.715261] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=33][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:56:59.715278] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=16][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:56:59.715285] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63400-0-0] [lt=5][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:56:59.747981] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=3544) [2025-03-17 09:56:59.778524] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=24] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:56:59.780654] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.781184] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703079}], scn_array=[cnt:1, {val:1742176619656135001, v:0}], curr_size=122, accum_size=488, time_cost=21905) [2025-03-17 09:56:59.781288] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463512347}], scn_array=[cnt:1, {val:1742176619747568000, v:0}], curr_size=587, accum_size=3422, time_cost=33302) [2025-03-17 09:56:59.782331] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=49][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415224}], scn_array=[cnt:1, {val:1742176619656135001, v:0}], curr_size=122, accum_size=610, time_cost=23029) [2025-03-17 09:56:59.782384] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=15235, avg_log_submit_to_slide_time=15253) [2025-03-17 09:56:59.782447] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:56:59.796979] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=39] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40317701324, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.802086] INFO [SERVER] get_dbms_sched_job_infos_in_tenant (ob_dbms_sched_table_operator.cpp:682) [3539266][T1004_DBMSSched][T1004][Y0-0000000000000000-0-0] [lt=23] failed to get result(ret=-4008) [2025-03-17 09:56:59.802182] INFO [SERVER] check_new_jobs (ob_dbms_sched_job_master.cpp:528) [3539266][T1004_DBMSSched][T1004][Y0-0000000000000000-0-0] [lt=24] check new jobs(ret=0, tenant_id=1004, is_oracle_tenant=false, job_infos=[{tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742176464566241, start_date_:1741596864436800, last_date_:1742176464437983, this_date_:0, next_date_:1742177364436800, end_date_:64060560000000000, total_:78290668, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:5, job_name_:"FRIDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741960807199451, start_date_:1741960800000000, last_date_:1741960800001045, this_date_:0, next_date_:1742565600000000, end_date_:64060560000000000, total_:7198406, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(14400000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:14400, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:1, job_name_:"MONDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741615230254743, start_date_:1741615200000000, last_date_:1741615200001124, this_date_:0, next_date_:1742220000000000, end_date_:64060560000000000, total_:30253619, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(14400000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:14400, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114364542592, start_date_:1741682364436700, last_date_:1742114364437903, this_date_:0, next_date_:1742200764436700, end_date_:64060560000000000, total_:769378, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:6, job_name_:"SATURDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741989602301309, start_date_:1741989600000000, last_date_:1741989600001026, this_date_:0, next_date_:1742594400000000, end_date_:64060560000000000, total_:2300283, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742144400026360, start_date_:1741626000000000, last_date_:1742144400001510, this_date_:0, next_date_:1742230800000000, end_date_:64060560000000000, total_:187622, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:7, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076002000175, start_date_:1742076000000000, last_date_:1742076000000980, this_date_:0, next_date_:1742680800000000, end_date_:64060560000000000, total_:1999195, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:4, job_name_:"THURSDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741874403035716, start_date_:1741874400000000, last_date_:1741874400001270, this_date_:0, next_date_:1742479200000000, end_date_:64060560000000000, total_:3034446, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(14400000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:14400, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:2, job_name_:"TUESDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741701602853199, start_date_:1741701600000000, last_date_:1741701600003726, this_date_:0, next_date_:1742306400000000, end_date_:64060560000000000, total_:2849473, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(14400000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:14400, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}, {tenant_id_:1004, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:3, job_name_:"WEDNESDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1741788006160413, start_date_:1741788000000000, last_date_:1741788000001192, this_date_:0, next_date_:1742392800000000, end_date_:64060560000000000, total_:6159221, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(14400000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:14400, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}]) [2025-03-17 09:56:59.802260] INFO [SERVER] check_tenant (ob_dbms_sched_job_master.cpp:517) [3539266][T1004_DBMSSched][T1004][Y0-0000000000000000-0-0] [lt=77] check all tenants(ret=0) [2025-03-17 09:56:59.828918] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=52][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859698309}], scn_array=[cnt:1, {val:1742176619795608000, v:0}], curr_size=588, accum_size=2546, time_cost=33183) [2025-03-17 09:56:59.853923] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858F9-0-0] [lt=23] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1742176619853906}) [2025-03-17 09:56:59.853951] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3189-0-0] [lt=19] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.853946] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=54] new mysql sessid created(conn.sessid_=3222507924, support_ssl=false) [2025-03-17 09:56:59.854002] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222507924, sess.client_addr_="127.0.0.1:9342") [2025-03-17 09:56:59.854022] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=14] accept one succ(*s={this:0x7f79906aadb0, session_id:3222507924, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:252}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:56:59.854064] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118875][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEAC9D-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11017, add_timestamp:1742176619853910}) [2025-03-17 09:56:59.854104] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=20] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:56:59.854135] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117619][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=12] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:56:59.854160] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:292) [117619][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=25] mysql login queue(mysql_queue->queue_.size()=1) [2025-03-17 09:56:59.854244] INFO load_privilege_info (obmp_connect.cpp:582) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A0-0-0] [lt=26] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:56:59.854716] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3189-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.854810] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3189-0-0] [lt=93] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1742176619853906}, locations=[{table_id:11017, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619854700, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.854866] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318A-0-0] [lt=34] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11017, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.855057] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318A-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11017, locations=[{table_id:11017, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.855087] INFO [SERVER] process (obmp_connect.cpp:523) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A0-0-0] [lt=17] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222507924, proxy_sessid=0, client_sessid=3222507924, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:56:59.855089] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318A-0-0] [lt=31] success to process renew task(task={tenant_id:1, table_id:11017, add_timestamp:1742176619853910}, locations=[{table_id:11017, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}, {table_id:11017, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855051, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.855334] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E7552FC-0-0] [lt=6] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11095, add_timestamp:1742176619855326}) [2025-03-17 09:56:59.855347] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318B-0-0] [lt=36] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11095, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.855609] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318B-0-0] [lt=25] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11095, locations=[{table_id:11095, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.855650] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318B-0-0] [lt=40] success to process renew task(task={tenant_id:1, table_id:11095, add_timestamp:1742176619855326}, locations=[{table_id:11095, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}, {table_id:11095, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619855597, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.856614] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:728) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=30] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=12000000, replace_num=100000) [2025-03-17 09:56:59.856645] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=31] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=10) [2025-03-17 09:56:59.857904] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E7552FD-0-0] [lt=20] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11008, add_timestamp:1742176619857896}) [2025-03-17 09:56:59.857923] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=55] new mysql sessid created(conn.sessid_=3222509039, support_ssl=false) [2025-03-17 09:56:59.857920] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318C-0-0] [lt=30] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11008, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.857969] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222509039, sess.client_addr_="127.0.0.1:9346") [2025-03-17 09:56:59.857986] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=13] accept one succ(*s={this:0x7f7af9aa8db0, session_id:3222509039, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:273}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:56:59.858028] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=20] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:56:59.858065] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117622][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=16] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:56:59.858138] INFO load_privilege_info (obmp_connect.cpp:582) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED6835F-0-0] [lt=33] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:56:59.858135] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318C-0-0] [lt=26] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11008, locations=[{table_id:11008, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.858183] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318C-0-0] [lt=48] success to process renew task(task={tenant_id:1, table_id:11008, add_timestamp:1742176619857896}, locations=[{table_id:11008, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}, {table_id:11008, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619858123, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.858870] INFO [SERVER] process (obmp_connect.cpp:523) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED6835F-0-0] [lt=38] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222509039, proxy_sessid=0, client_sessid=3222509039, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:56:59.859084] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FA-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11003, add_timestamp:1742176619859070}) [2025-03-17 09:56:59.859115] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318D-0-0] [lt=17] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11003, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.859300] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=2, this=0x7f79ae329470, extra_info="") [2025-03-17 09:56:59.859318] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318D-0-0] [lt=21] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11003, locations=[{table_id:11003, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.859335] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:56:59.859344] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318D-0-0] [lt=25] success to process renew task(task={tenant_id:1, table_id:11003, add_timestamp:1742176619859070}, locations=[{table_id:11003, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}, {table_id:11003, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619859309, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.859470] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=23] QSched: |cfg: root:1,x,0 net_in:1,1249M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:280K/66:15562 disk:280K/66:15562 t1m2:76K/18:11836 t1m2g0:76K/18:11836 t1004m2:96K/23:18209 t1004m2g0:96K/23:18209 t1003m2:108K/25:15810 t1003m2g0:108K/25:15810 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:56:59.862034] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:103) [118875][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEAC9D-0-0] [lt=21][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:56:59.862073] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_event.cpp:103) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858F9-0-0] [lt=29][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:56:59.862339] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9aa8db0, session_id:3222509039, trace_id:YB420A0B093E-00062FB937B858FD-0-0, sql_handling_stage:10, sql_initiative_shutdown:false, reader:{fd:273}, err:5, last_decode_time:1742176619861685, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:800, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:56:59.862364] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=23][errcode=0] server close connection(sessid=3222509039, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:59.862385] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=40] kill and revert session(conn.sessid_=3222509039, proxy_sessid=0, ret=0) [2025-03-17 09:56:59.862399] INFO [SERVER] process (obmp_quit.h:56) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=33] quit [2025-03-17 09:56:59.862414] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=13][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:56:59.862456] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=42][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:56:59.862466] WDIAG [SERVER] response (obmp_base.cpp:87) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=10][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:56:59.862476] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=9][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:56:59.862487] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=9][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.862494] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FD-0-0] [lt=5][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.862502] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118872][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.864545] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=9] can close safely, do destroy(*s={this:0x7f7af9aa8db0, session_id:3222509039, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:273}, err:5, last_decode_time:1742176619861685, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:805, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:56:59.864581] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=22] unlock session of tenant(conn.sessid_=3222509039, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:56:59.864630] INFO [SERVER] run (obmp_disconnect.cpp:83) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=13] free session successfully(sessid=3222509039) [2025-03-17 09:56:59.864593] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] connection close(sessid=3222509039, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB937B858FC-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:20, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:98, rec_:98, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:3, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:253, rec_:253, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[8]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[11]:{send:obp_mysql_header_{pkt_num_:1, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:336, rec_:336, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[14]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{send:obp_mysql_header_{pkt_num_:2, seq_:8}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[18]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:56:59.864656] INFO [SERVER] run (obmp_disconnect.cpp:90) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=25] mark session id unused(sessid=3222509039) [2025-03-17 09:56:59.864727] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FE-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11021, add_timestamp:1742176619864714}) [2025-03-17 09:56:59.864738] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318E-0-0] [lt=22] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11021, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.864968] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318E-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11021, locations=[{table_id:11021, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.865015] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318E-0-0] [lt=46] success to process renew task(task={tenant_id:1, table_id:11021, add_timestamp:1742176619864714}, locations=[{table_id:11021, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}, {table_id:11021, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619864951, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.877910] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FE-0-0] [lt=29][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:56:59.879723] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=58][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9b46db0, session_id:3222518387, trace_id:YB420A0B093E-00062FB937B858FF-0-0, sql_handling_stage:2, sql_initiative_shutdown:false, reader:{fd:251}, err:5, last_decode_time:1742176619878671, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:3716, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:56:59.879774] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=35] kill and revert session(conn.sessid_=3222518387, proxy_sessid=0, ret=0) [2025-03-17 09:56:59.879834] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=37][errcode=0] server close connection(sessid=3222518387, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:59.879855] INFO [SERVER] process (obmp_quit.h:56) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=20] quit [2025-03-17 09:56:59.879863] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=7][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:56:59.879871] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=8][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:56:59.879878] WDIAG [SERVER] response (obmp_base.cpp:87) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=7][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:56:59.879883] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=4][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:56:59.879896] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=12][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.879902] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B858FF-0-0] [lt=5][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.879909] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118872][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:56:59.879963] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E7552FE-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12254, add_timestamp:1742176619879955}) [2025-03-17 09:56:59.879981] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318F-0-0] [lt=35] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12254, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.880234] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318F-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12254, locations=[{table_id:12254, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.880267] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E318F-0-0] [lt=32] success to process renew task(task={tenant_id:1, table_id:12254, add_timestamp:1742176619879955}, locations=[{table_id:12254, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}, {table_id:12254, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619880216, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.880732] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.881168] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415346}], scn_array=[cnt:1, {val:1742176619656135002, v:0}], curr_size=122, accum_size=732, time_cost=21788) [2025-03-17 09:56:59.881371] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16392, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:56:59.881391] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703201}], scn_array=[cnt:1, {val:1742176619656135002, v:0}], curr_size=122, accum_size=610, time_cost=22030) [2025-03-17 09:56:59.881450] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=51] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=29, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:56:59.881480] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=22] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:56:59.881506] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16515, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:56:59.881522] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=32, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:56:59.882154] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=12] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2025-03-17 09:56:59.883254] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=15] get number(ret=0, service_type_=0, range=1, base_id=1742176619882952000, start_id=1742176619882952000, end_id=1742176619882952001) [2025-03-17 09:56:59.883733] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118875][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEAC9E-0-0] [lt=22][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:56:59.894645] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E7552FF-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11018, add_timestamp:1742176619894628}) [2025-03-17 09:56:59.894669] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3190-0-0] [lt=20] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11018, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.894705] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E7552FF-0-0] [lt=31] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11092, add_timestamp:1742176619894703}) [2025-03-17 09:56:59.894915] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3190-0-0] [lt=27] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11018, locations=[{table_id:11018, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.894945] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3190-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:11018, add_timestamp:1742176619894628}, locations=[{table_id:11018, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}, {table_id:11018, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619894902, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.894973] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3191-0-0] [lt=20] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11092, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:56:59.895251] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3191-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11092, locations=[{table_id:11092, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.895297] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3191-0-0] [lt=45] success to process renew task(task={tenant_id:1, table_id:11092, add_timestamp:1742176619894703}, locations=[{table_id:11092, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}, {table_id:11092, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619895238, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:56:59.900640] INFO [SERVER] init (ob_all_virtual_tenant_parameter_stat.cpp:65) [118838][T1_L1_G10000][T1][YB420A0B093E-00062FB95E7552FF-0-0] [lt=7] sys tenant show all local tenant parameter(effective_tenant_id_=1, tenant_id_list_=[1, 1003, 1004]) [2025-03-17 09:56:59.920694] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859698897}], scn_array=[cnt:1, {val:1742176619882952000, v:0}], curr_size=545, accum_size=3091, time_cost=37347) [2025-03-17 09:56:59.927334] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-4006] Rpc proxy not inited(ret=-4006) [2025-03-17 09:56:59.930104] WDIAG [SHARE] get_restore_shared_storage_limit (ob_backup_connectivity.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=36][errcode=-4018] no exist row(ret=-4018, sql=SELECT max_iops, max_bandwidth FROM __all_backup_storage_info WHERE tenant_id = 1004 AND dest_id = '1') [2025-03-17 09:56:59.932492] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=8] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=9208, avg_group_log_size=575) [2025-03-17 09:56:59.933370] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=43][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933397] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=26][errcode=0] update shared device limit failed(ret=0, i=2, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933410] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=12][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933443] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=33][errcode=0] update shared device limit failed(ret=0, i=5, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933456] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=12][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933467] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=11][errcode=0] update shared device limit failed(ret=0, i=8, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933478] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=10][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933486] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=8][errcode=0] update shared device limit failed(ret=0, i=11, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933494] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933499] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=5][errcode=0] update shared device limit failed(ret=0, i=14, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933504] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=5][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933512] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=17, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933520] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933527] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=20, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933535] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933542] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=23, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933550] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933557] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=26, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933565] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933572] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=29, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933580] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933587] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=32, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933593] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=5][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933600] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=35, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933608] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933616] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=8][errcode=0] update shared device limit failed(ret=0, i=38, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:56:59.933623] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:56:59.933631] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95067DE58-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=41, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:56:59.933729] INFO pn_ratelimit (group.c:653) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=10] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2025-03-17 09:56:59.933782] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117409][TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=41] trace max work area(auto_calc=true, tenant_max_memory_limit=1717986918, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=85899345, max_wa_memory_size=85899345, tmp_max_wa_memory_size=85899345, pre_mem_target=85899345, remain_memory_size=85899345, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=1428951040, washable_size=-2, max_workarea_memory_size=85899345, max_tenant_memory_size=289035878, wash_ratio=6, tenant_id=1003) [2025-03-17 09:56:59.933817] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=35][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:56:59.933837] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=18][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:56:59.933858] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117409][TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=43] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=10737418, calc_global_bound_size=1048576000000, mem_target_=85899345, wa_max_memory_size=85899345, profile_cnt_=0, total_memory_size=0, max_wa_size=85899345, avg_bound_size=10737418, max_bound_size=10737418, min_bound_size_=1048576) [2025-03-17 09:56:59.933880] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117409][TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=20] timer to calc global bound size(ret=0, global_bound_size_=10737418, manual_calc_cnt_=44644, drift_size_=0, pre_drift_size=0, wa_max_memory_size=85899345, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=10737418, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=85899345, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2025-03-17 09:56:59.933934] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117409][TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=18] trace max work area(auto_calc=true, tenant_max_memory_limit=15461882266, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=773094113, max_wa_memory_size=773094113, tmp_max_wa_memory_size=773094113, pre_mem_target=773094113, remain_memory_size=773094113, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=7391244288, washable_size=-2, max_workarea_memory_size=773094113, max_tenant_memory_size=8070637978, wash_ratio=6, tenant_id=1004) [2025-03-17 09:56:59.933990] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117409][TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=23] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=96636764, calc_global_bound_size=1048576000000, mem_target_=773094113, wa_max_memory_size=773094113, profile_cnt_=0, total_memory_size=0, max_wa_size=773094113, avg_bound_size=96636764, max_bound_size=96636764, min_bound_size_=1048576) [2025-03-17 09:56:59.934005] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117409][TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=14] timer to calc global bound size(ret=0, global_bound_size_=96636764, manual_calc_cnt_=492, drift_size_=0, pre_drift_size=0, wa_max_memory_size=773094113, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=96636764, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=773094113, auto_calc=true, sql_mem_callback_.get_total_dump_size()=69192448) [2025-03-17 09:56:59.938770] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=7] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312802662513, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:59.938805] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=31][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:56:59.938816] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=11] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7ad38a9970) [2025-03-17 09:56:59.938824] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=8] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312802662513, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:56:59.946678] INFO [LIB] stat (utility.h:1210) [175477][T1_L0_G5][T1][YB420A0B0940-00063024191DDF15-0-0] [lt=31] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:56:59.946720] INFO [LIB] stat (utility.h:1210) [175477][T1_L0_G5][T1][YB420A0B0940-00063024191DDF15-0-0] [lt=29] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:56:59.946730] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [175477][T1_L0_G5][T1][YB420A0B0940-00063024191DDF15-0-0] [lt=9] submit log success(service_type_=0, last_id=1742176636588679000, limited_id=1742176636588679000) [2025-03-17 09:56:59.946738] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [175477][T1_L0_G5][T1][YB420A0B0940-00063024191DDF15-0-0] [lt=8] submit log with lock success(service_type_=0, last_id=1742176636588679000, limited_id=1742176636588679000) [2025-03-17 09:56:59.946825] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:109) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=43] is logging [2025-03-17 09:56:59.946847] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:57) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=21] submit log with lock(ret=-4023, service_type_=0, last_id=1742176636588679000, limited_id=1742176636588679000) [2025-03-17 09:56:59.957137] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12142, add_timestamp:1742176619957123}) [2025-03-17 09:56:59.957182] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3192-0-0] [lt=34] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12142, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957166, sql_renew_time:1742176619957166, is_mark_fail:false}]) [2025-03-17 09:56:59.957202] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=49] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12217, add_timestamp:1742176619957199}) [2025-03-17 09:56:59.957212] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3192-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:12142, add_timestamp:1742176619957123}, locations=[{table_id:12142, partition_id:12142, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957166, sql_renew_time:1742176619957166, is_mark_fail:false}]) [2025-03-17 09:56:59.957217] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12068, add_timestamp:1742176619957216}) [2025-03-17 09:56:59.957233] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11035, add_timestamp:1742176619957232}) [2025-03-17 09:56:59.957233] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3193-0-0] [lt=14] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12217, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957230, sql_renew_time:1742176619957230, is_mark_fail:false}]) [2025-03-17 09:56:59.957244] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3193-0-0] [lt=11] success to process renew task(task={tenant_id:1, table_id:12217, add_timestamp:1742176619957199}, locations=[{table_id:12217, partition_id:12217, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957230, sql_renew_time:1742176619957230, is_mark_fail:false}]) [2025-03-17 09:56:59.957248] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=10] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12072, add_timestamp:1742176619957247}) [2025-03-17 09:56:59.957260] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3194-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12068, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957257, sql_renew_time:1742176619957257, is_mark_fail:false}]) [2025-03-17 09:56:59.957267] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3194-0-0] [lt=7] success to process renew task(task={tenant_id:1, table_id:12068, add_timestamp:1742176619957216}, locations=[{table_id:12068, partition_id:12068, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957257, sql_renew_time:1742176619957257, is_mark_fail:false}]) [2025-03-17 09:56:59.957281] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3195-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11035, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957278, sql_renew_time:1742176619957278, is_mark_fail:false}]) [2025-03-17 09:56:59.957291] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3195-0-0] [lt=10] success to process renew task(task={tenant_id:1, table_id:11035, add_timestamp:1742176619957232}, locations=[{table_id:11035, partition_id:11035, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957278, sql_renew_time:1742176619957278, is_mark_fail:false}]) [2025-03-17 09:56:59.957306] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3196-0-0] [lt=9] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12072, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957303, sql_renew_time:1742176619957303, is_mark_fail:false}]) [2025-03-17 09:56:59.957315] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3196-0-0] [lt=9] success to process renew task(task={tenant_id:1, table_id:12072, add_timestamp:1742176619957247}, locations=[{table_id:12072, partition_id:12072, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176619957303, sql_renew_time:1742176619957303, is_mark_fail:false}]) [2025-03-17 09:56:59.959383] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:56:59.964897] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859699442}], scn_array=[cnt:1, {val:1742176619932099000, v:0}], curr_size=588, accum_size=3679, time_cost=32396) [2025-03-17 09:56:59.965095] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=23] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=14247, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:56:59.965128] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463513056}], scn_array=[cnt:1, {val:1742176619945902001, v:0}], curr_size=128, accum_size=3672, time_cost=18368) [2025-03-17 09:56:59.965169] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=3672) [2025-03-17 09:56:59.965195] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=24, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:56:59.965218] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=44] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=3672) [2025-03-17 09:56:59.965276] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:56:59.965296] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=14361, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:56:59.965323] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=43, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:56:59.965313] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:846) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=4] update id meta(ret=0, service_type=0, limited_id=1742176636588679000, latest_scn={val:1742176619945902001, v:0}, *this={tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636588679000, latest_log_ts:{val:1742176619945902001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}) [2025-03-17 09:56:59.965376] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=62] handle submit callback(service_type_=0, success=true, used_ts=18646, log_ts={val:1742176619945902001, v:0}, limited_id=1742176636588679000) [2025-03-17 09:56:59.965389] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:564) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=12] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2025-03-17 09:56:59.965397] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:602) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=8] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2025-03-17 09:56:59.980492] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415468}], scn_array=[cnt:1, {val:1742176619656135003, v:0}], curr_size=122, accum_size=854, time_cost=21033) [2025-03-17 09:56:59.980633] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=298, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.980681] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=46][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1656131) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:56:59.980701] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=19][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:56:59.980710] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7de8, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95E755300-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619979287, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:1302, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.980737] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703323}], scn_array=[cnt:1, {val:1742176619656135003, v:0}], curr_size=122, accum_size=732, time_cost=21352) [2025-03-17 09:56:59.980758] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=42][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7de8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95E755300-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619979287, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:1302, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:1742176619980754}}, part_id:-1}) [2025-03-17 09:56:59.980764] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=9][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619979287, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:1302, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:162, end_used:-38}}) [2025-03-17 09:56:59.980813] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=45][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.980806] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:56:59.980825] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=9][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:56:59.981117] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=34][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=298, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:56:59.981135] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=18][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:56:59.981161] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=26][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a22815543, src_buf_size=298) [2025-03-17 09:56:59.981168] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7fe0, ref_cnt_:1, raw_buf_:0x7f7a22814150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95E755300-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619980840, enqueue_used:1, dequeue_used:16, submit_used:0, return_used:238, callback_enqueue_used:2, callback_dequeue_used:19, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:56:59.981201] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=29][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7fe0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95E755300-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619980840, enqueue_used:1, dequeue_used:16, submit_used:0, return_used:238, callback_enqueue_used:2, callback_dequeue_used:19, callback_finish_used:-1, end_used:1742176619981198}}, part_id:-1}) [2025-03-17 09:56:59.981208] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=11][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1656131, size_:298, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79957b8080, time_log:{begin_ts:1742176619980840, enqueue_used:1, dequeue_used:16, submit_used:0, return_used:238, callback_enqueue_used:2, callback_dequeue_used:19, callback_finish_used:111, end_used:-29}}) [2025-03-17 09:56:59.981246] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=34][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:56:59.981260] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=12][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1656131, size:298, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f79957b8080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.981285] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=24][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1656131, size:298, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f79957b8080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:56:59.981303] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=16][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1656131, size:298, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:56:59.981317] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=14][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f790c9b6bab, tablet_:0x7f790c9b40a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.981340] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=22][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f790c9b6bab, tablet_:0x7f790c9b40a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f79740b9080, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:6, rowkey_vector:{col_cnt:5, row_cnt:1, is_datum_vectors:-1, columns:0x7f790c9b7128, last_rowkey:0x7f790c9b72e8, discrete_rowkey_array:0x7f790c9b7440, last_rowkey_:{datum_cnt:5, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f790c9b71d8, hex: 0000000000000000, int: 0}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f790c9b7210, hex: 0502000000000000, int: 517}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f790c9b7248, hex: 0502000000000000, int: 517}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f790c9b72a8, hex: 77A121C85910D3E7, int: -1742030653066469001}, idx=4:{len: 8, flag: 0, null: 0, ptr: 0x7f790c9b72c0, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f790c9b70ad, ps_node_array:0x7f790c9b70e8, data_buf:0x7f790c9b70ad, data_buf_size:1175}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f790c9b6bab, tablet_:0x7f790c9b40a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:56:59.981388] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=47][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=2, prefetched_cnt=0) [2025-03-17 09:56:59.981397] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=8][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:56:59.981406] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=8][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:56:59.981418] WDIAG [STORAGE] inner_open (ob_sstable_multi_version_row_iterator.cpp:162) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=7][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:56:59.981462] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=43][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:328, tablet_id:{id:328}, cg_idx:0, read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:3, column_cnt:28}, cols_desc:{ObIArray:[cnt:5, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f790c9b4668, count:5, allocator:null, capacity:5, init_cnt:5}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:3, column_cnt:28}}, out_cols_project:[cnt:2, 1, 2], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f791d010c50, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:2, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f791d012da0}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176719879012, ls_id:{id:1}, tablet_id:{id:328}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f79a5be6e90, limit_param:null, stmt_allocator:0x7f791d0107f8, allocator:0x7f791d010830, range_allocator:null, table_scan_stat:0x7f791d0123a8, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741716004303628000, snapshot_version:1742176619932099000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f791d014b40, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:56:59.981578] WDIAG [STORAGE] scan (ob_sstable.cpp:427) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=115][errcode=-4016] Fail to open row scanner(ret=-4016, param={table_id:328, tablet_id:{id:328}, cg_idx:0, read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:3, column_cnt:28}, cols_desc:{ObIArray:[cnt:5, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f790c9b4668, count:5, allocator:null, capacity:5, init_cnt:5}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:3, column_cnt:28}}, out_cols_project:[cnt:2, 1, 2], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f791d010c50, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:2, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f791d012da0}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176719879012, ls_id:{id:1}, tablet_id:{id:328}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f79a5be6e90, limit_param:null, stmt_allocator:0x7f791d0107f8, allocator:0x7f791d010830, range_allocator:null, table_scan_stat:0x7f791d0123a8, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741716004303628000, snapshot_version:1742176619932099000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f791d014b40, sample_filter:null, mview_scan_info:NULL}, key_range={start_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f791d013150, hex: 0000000000000000, int: 0}, idx=1:MIN_OBJ, idx=2:MIN_OBJ], store_rowkey:{"BIGINT":0},MIN,MIN}, end_key:{datum_cnt:3, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f791d013200, hex: 0000000000000000, int: 0}, idx=1:MAX_OBJ, idx=2:MAX_OBJ], store_rowkey:{"BIGINT":0},MAX,MAX}, group_idx:0, border_flag:{flag:0}}, *this={ObITable:{this:0x7f790c9b6bab, key:{tablet_id:{id:328}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1742016323170786000, v:0}, end_scn:{val:1742045627257879000, v:0}}}, ref_cnt:0, upper_trans_version:1742039653140665000, timestamp:0}, this:0x7f790c9b6bab, addr:{[ver=1,mode=0,seq=39199][2nd=39200][3rd=0][offset=1818053,size=955,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1654784, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:6, occupy_size:1347, data_checksum:0, max_merged_trans_version:1742039653140665000, upper_trans_version:1742039653140665000, filled_tx_scn:{val:1742045627257879000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:6, occupy_size_:1347, original_size_:866, data_checksum_:1754394480, index_type_:0, rowkey_column_count_:5, column_cnt_:28, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741595949178112, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1742039653140665000, max_merged_trans_version_:1742039653140665000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1742045627257879000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=298,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f790c9b6f4b, block_data:{buf:0x7f790c9b6f0b, size:64, extra_buf:0x7f790c9b7075, extra_size:1175, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=278,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f790c9b750c, block_data:{buf:0x7f790c9b750c, size:278, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f790c9b7622, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1654784, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2025-03-17 09:56:59.981721] WDIAG [STORAGE] construct_iters (ob_multiple_scan_merge.cpp:198) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=140][errcode=-4016] Fail to get iterator(ret=-4016, i=3, *iter_pram={table_id:328, tablet_id:{id:328}, cg_idx:0, read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:3, column_cnt:28}, cols_desc:{ObIArray:[cnt:5, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f790c9b4668, count:5, allocator:null, capacity:5, init_cnt:5}, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:3, column_cnt:28}}, out_cols_project:[cnt:2, 1, 2], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f791d010c50, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:2, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f791d012da0}) [2025-03-17 09:56:59.981788] WDIAG [STORAGE] open (ob_multiple_scan_merge.cpp:60) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=65][errcode=-4016] fail to construct iters(ret=-4016) [2025-03-17 09:56:59.981796] WDIAG [STORAGE] init_and_open_scan_merge_iter_ (ob_table_scan_iterator.cpp:651) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=8][errcode=-4016] Fail to open multiple merge iterator(ret=-4016) [2025-03-17 09:56:59.981804] WDIAG [STORAGE] open_iter (ob_table_scan_iterator.cpp:558) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=7][errcode=-4016] init and open scan merge iterator failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:56:59.981813] WDIAG [STORAGE] init (ob_table_scan_iterator.cpp:355) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=8][errcode=-4016] fail to open iter(ret=-4016, cached_iter_node_=NULL, *this={}) [2025-03-17 09:56:59.981821] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2373) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=7][errcode=-4016] failed to init table scan iterator, (ret=-4016) [2025-03-17 09:56:59.981828] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2379) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=6][errcode=-4016] failed to do table scan(ret=-4016, param={tablet_id:{id:328}, ls_id:{id:1}, column_ids:[19, 20], index_id:328, key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,({"BIGINT":0},MIN,MIN;{"BIGINT":0},MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176719879012, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:2, schema_version:1741595949178112, query_begin_schema_version:1741597433156656, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f791d012ad0, valid:true, source:2, core:{version:{val:1742176619932099000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:328, output_projector:[cnt:2, 1, 2], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:2, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176619977121, need_update_tablet_param:false}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2025-03-17 09:56:59.981918] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:674) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=90][errcode=-4016] failed to do table scan(ret=-4016, &iter=0x7f79a5be6210, param={tablet_id:{id:328}, ls_id:{id:1}, column_ids:[19, 20], index_id:328, key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,({"BIGINT":0},MIN,MIN;{"BIGINT":0},MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176719879012, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:2, schema_version:1741595949178112, query_begin_schema_version:1741597433156656, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f791d012ad0, valid:true, source:2, core:{version:{val:1742176619932099000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:328, output_projector:[cnt:2, 1, 2], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:2, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176619977121, need_update_tablet_param:false}) [2025-03-17 09:56:59.982006] WDIAG [STORAGE] table_scan (ob_access_service.cpp:371) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=87][errcode=-4016] Fail to scan table, (ret=-4016, ls_id={id:1}, param={tablet_id:{id:328}, ls_id:{id:1}, column_ids:[19, 20], index_id:328, key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,({"BIGINT":0},MIN,MIN;{"BIGINT":0},MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176719879012, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:2, schema_version:1741595949178112, query_begin_schema_version:1741597433156656, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f791d012ad0, valid:true, source:2, core:{version:{val:1742176619932099000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:328, output_projector:[cnt:2, 1, 2], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:2, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176619977121, need_update_tablet_param:false}) [2025-03-17 09:56:59.982078] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_iter.cpp:83) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=71][errcode=-4016] fail to scan table(scan_param={tablet_id:{id:328}, ls_id:{id:1}, column_ids:[19, 20], index_id:328, key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,({"BIGINT":0},MIN,MIN;{"BIGINT":0},MAX,MAX)"}], ss_key_ranges:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176719879012, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:2, schema_version:1741595949178112, query_begin_schema_version:1741597433156656, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f791d012ad0, valid:true, source:2, core:{version:{val:1742176619932099000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:328, output_projector:[cnt:2, 1, 2], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:2, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176619977121, need_update_tablet_param:false}, ret=-4016) [2025-03-17 09:56:59.982167] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:569) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=88][errcode=-4016] iter tree failed to do table scan(ret=-4016) [2025-03-17 09:56:59.982173] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=5][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:56:59.982181] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:102) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=6][errcode=-4016] start das task failed(ret=-4016, *task_wrapper.at(i)={parent:{tenant_id:1003, task_id:56388578, op_type:1, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7ad17f2520, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520320, assoc_session_id:3222520320, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176619978185, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176619977121, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, snapshot:{this:0x7f7a06c507d8, valid:true, source:2, core:{version:{val:1742176619932099000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, tablet_id:{id:328}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:328}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:328, related_table_ids:[cnt:0], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:1}, in_retry:0, partition_id:328, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:, plan_line_id:0}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:328, access_column_ids:[cnt:2, 19, 20], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1741595949178112, table_param:{table_id:328, output_projector:[cnt:2, 1, 2], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:2, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:26, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3d0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f78a4ffe3e0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f78a4ffe3a0, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:3, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f78a4ffe410, count:3, allocator:0x7f789879af30, capacity:3, init_cnt:3}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, pd_expr_spec:{calc_exprs:[cnt:0], access_exprs:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:16, pushdown_filters:[cnt:0], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:2, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:200, res_buf_off:1366, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1206, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:878, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc168}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:1494, dyn_buf_header_offset:1238, res_buf_len:8, eval_flags_off:1214, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:943, eval_vector_func:null, local_session_var_id:-1, this:0x7f78a4ffc250}], is_get:false, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0, rowkey_exprs:[cnt:0], table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, doc_id_idx:-1, vec_vid_idx:-1, index_merge_idx:-1, pre_query_range:[], is_index_merge:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:1741597433156656, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1742176719879012, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tsc_monitor_info:{*io_read_bytes_:298*ssstore_read_bytes_:0*ssstore_read_row_cnt_:0*memstore_read_row_cnt_:0}, key_ranges:[], ss_key_ranges:[], mbr_filters:[], scan_op_id:18446744073709551615, scan_rows_size:-1, das_tasks_key:{fake_unique_id_:0, timestamp_:0}}, scan_range:[{"range":"table_id:328,group_idx:0,index_ordered_idx:0,({"BIGINT":0},MIN,MIN;{"BIGINT":0},MAX,MAX)"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}}) [2025-03-17 09:56:59.982340] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=158][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:56:59.982350] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=9][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:56:59.982354] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:260) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=4][errcode=-4016] failed to execute all das task(ret=-4016) [2025-03-17 09:56:59.982358] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2699) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=4][errcode=-4016] execute all das scan task failed(ret=-4016) [2025-03-17 09:56:59.982363] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1822) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=4][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:56:59.982455] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [118869][T1_L0_G10000][T1003][YB420A0B093E-00062FB95E755300-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ad17f2520, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520320, assoc_session_id:3222520320, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176619978185, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176619977121, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7b00e5ef58, id:3222520320, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ad17f2520}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:56:59.982716] INFO [SERVER] send_error_packet (obmp_packet_sender.cpp:380) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755300-0-0] [lt=0] sending error packet(ob_error=-4016, client error=-4016, extra_err_info=NULL, lbt()="0x810107f 0x10192510 0x10141ce3 0x790e5a0 0x78c143b 0x78b88e4 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:56:59.983379] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463513184}], scn_array=[cnt:1, {val:1742176619945902002, v:0}], curr_size=587, accum_size=587, time_cost=18106) [2025-03-17 09:56:59.983508] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1742176619982456000, start_id=1742176619982456000, end_id=1742176619982456001) [2025-03-17 09:56:59.986619] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=23] ====== checkpoint timer task ====== [2025-03-17 09:56:59.986658] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=20] get_max_applied_scn(scn={val:1742176619882952000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:4859700030}, palf_committed_end_scn_:{val:1742176619932099000, v:0}, last_check_scn_:{val:1742176619932099000, v:0}, max_applied_cb_scn_:{val:1742176619882952000, v:0}}) [2025-03-17 09:56:59.986676] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=16] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595943665555, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79aaff7a70, log_storage_:{log_tail:{lsn:4859700030}, readable_log_tail:{lsn:4859700030}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4831543296}, min_scn:{val:1742173162720054001, v:0}, curr_block_id:72, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003/1/log", dir_fd:268, min_block_id:12, max_block_id:73, curr_writable_block_id:72}, logical_block_size_:67104768, curr_block_writable_size_:38948034, block_header_serialize_buf_:0x7f79aaff8738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:56:59.987404] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=72] succ to get rec scn(*this={this:0x7f79c0804030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:2, v:0}, max_applied_start_working_ts_:{val:2, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595943500009, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:995}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:59.987454] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=49] tx ctx memtable get rec scn(this={ObITable:{this:0x7f798c698080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1741716004436780, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f798c698080, snapshot_version:{val:1741716004436780, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:56:59.987549] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=23] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742157237104149001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742157237104149001, v:0}}, }, *this={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:56:59.987607] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=56] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1742157237104149001, v:0}, ls_id_={id:1}) [2025-03-17 09:56:59.987645] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=33] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1, v:0}) [2025-03-17 09:56:59.987655] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=0, rec_log_ts={val:1742176619532124001, v:0}) [2025-03-17 09:56:59.987663] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=1, rec_log_ts={val:1742160213403604001, v:0}) [2025-03-17 09:56:59.987670] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=6] get rec log scn(service_type_=2, rec_log_ts={val:1742161213638166001, v:0}) [2025-03-17 09:56:59.987680] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=6] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:59.987693] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=10] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:56:59.987703] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742157237104149001, v:0}, checkpoint_scn_in_ls_meta={val:1742157237104149001, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:56:59.987716] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4697333760}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4697333760}, lsn={lsn:4697333760}) [2025-03-17 09:56:59.987731] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4697333760}, ls->get_ls_id()={id:1}) [2025-03-17 09:56:59.987741] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2025-03-17 09:56:59.995760] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:56:59.997302] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40313507020, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:56:59.998812] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [117815][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=31] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f7b0abbc2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:57:00.002557] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA585-0-0] [lt=1][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.011667] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=30] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:57:00.011730] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=17] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:57:00.025752] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859700030}], scn_array=[cnt:1, {val:1742176619995225000, v:0}], curr_size=588, accum_size=4267, time_cost=29986) [2025-03-17 09:57:00.033869] INFO [STORAGE.TRANS] refresh_elr_tenant_config_ (ob_tx_elr_util.cpp:47) [117534][DBMS_JOB_MASTER][T1004][YB420A0B093E-00062FB7EEC54126-0-0] [lt=8] refresh tenant config success(tenant_id=1004, *this={last_refresh_ts:1742176620033604, can_tenant_elr:true}) [2025-03-17 09:57:00.056747] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=31] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=9) [2025-03-17 09:57:00.059493] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:00.063356] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=36] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:57:00.063415] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=11] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:00.070961] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC99-0-0] [lt=8] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12276, add_timestamp:1742176620070944}) [2025-03-17 09:57:00.070983] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3197-0-0] [lt=10] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12276, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.071222] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3197-0-0] [lt=22] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12276, locations=[{table_id:12276, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.071266] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3197-0-0] [lt=43] success to process renew task(task={tenant_id:1, table_id:12276, add_timestamp:1742176620070944}, locations=[{table_id:12276, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}, {table_id:12276, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620071209, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.072480] INFO [CLOG] is_in_sync (ob_log_handler.cpp:709) [117882][T1_L5_G0][T1][YB420A0B0940-00063020D00CE198-0-0] [lt=19] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2025-03-17 09:57:00.072608] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC9B-0-0] [lt=18] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12387, add_timestamp:1742176620072599}) [2025-03-17 09:57:00.072620] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3198-0-0] [lt=29] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12387, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.072815] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3198-0-0] [lt=19] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12387, locations=[{table_id:12387, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.072862] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3198-0-0] [lt=46] success to process renew task(task={tenant_id:1, table_id:12387, add_timestamp:1742176620072599}, locations=[{table_id:12387, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}, {table_id:12387, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620072803, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.074018] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC9C-0-0] [lt=32] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11037, add_timestamp:1742176620074009}) [2025-03-17 09:57:00.074025] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=12] new mysql sessid created(conn.sessid_=3222519507, support_ssl=false) [2025-03-17 09:57:00.074030] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3199-0-0] [lt=31] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11037, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.074058] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=16] sm conn init succ(conn.sessid_=3222519507, sess.client_addr_="127.0.0.1:9362") [2025-03-17 09:57:00.074076] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14] accept one succ(*s={this:0x7f7af9a98db0, session_id:3222519507, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:273}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.074102] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=20] can close safely, do destroy(*s={this:0x7f7af9b46db0, session_id:3222518387, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:251}, err:5, last_decode_time:1742176619878671, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:3721, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:57:00.074130] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14] unlock session of tenant(conn.sessid_=3222518387, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:57:00.074201] INFO [SERVER] run (obmp_disconnect.cpp:83) [117888][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=8] free session successfully(sessid=3222518387) [2025-03-17 09:57:00.074225] INFO [SERVER] run (obmp_disconnect.cpp:90) [117888][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=23] mark session id unused(sessid=3222518387) [2025-03-17 09:57:00.074142] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=11] connection close(sessid=3222518387, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB937B858FE-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:40, pkt_rec[8]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{pkt_num_:3, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[11]:{send:obp_mysql_header_{pkt_num_:4, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:491, rec_:491, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[14]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_mysql_header_{pkt_num_:5, seq_:7}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{send:obp_mysql_header_{pkt_num_:3, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[18]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{receive:obp_mysql_header_{len_:119, rec_:119, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[20]:{send:obp_mysql_header_{len_:27, seq_:2}, pkt_name:"PKT_ERR", obp_mysql_header_.is_send_:1}, pkt_rec[21]:{receive:obp_mysql_header_{len_:256, rec_:256, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[22]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[23]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[25]:{send:obp_mysql_header_{pkt_num_:4, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[26]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{receive:obp_mysql_header_{len_:281, rec_:281, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[28]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[31]:{send:obp_mysql_header_{pkt_num_:20, seq_:27}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[32]:{send:obp_mysql_header_{len_:9, seq_:28}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[33]:{receive:obp_mysql_header_{len_:1601, rec_:1842, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[34]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[35]:{send:obp_mysql_header_{pkt_num_:6, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[36]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[37]:{send:obp_mysql_header_{pkt_num_:579, seq_:70}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[38]:{send:obp_mysql_header_{len_:9, seq_:71}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[39]:{receive:obp_mysql_header_{len_:1, rec_:599, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:57:00.074231] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3199-0-0] [lt=14] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11037, locations=[{table_id:11037, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.074282] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=128] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.074276] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E3199-0-0] [lt=44] success to process renew task(task={tenant_id:1, table_id:11037, add_timestamp:1742176620074009}, locations=[{table_id:11037, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}, {table_id:11037, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620074216, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.074303] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117622][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=8] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.074358] INFO load_privilege_info (obmp_connect.cpp:582) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F0568498-0-0] [lt=39] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.075124] INFO [SERVER] process (obmp_connect.cpp:523) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F0568498-0-0] [lt=20] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222519507, proxy_sessid=0, client_sessid=3222519507, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.080457] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415590}], scn_array=[cnt:1, {val:1742176619656135004, v:0}], curr_size=122, accum_size=976, time_cost=20931) [2025-03-17 09:57:00.080867] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.081143] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA0-0-0] [lt=21] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11023, add_timestamp:1742176620081136}) [2025-03-17 09:57:00.081165] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319A-0-0] [lt=31] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11023, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.081324] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319A-0-0] [lt=14] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11023, locations=[{table_id:11023, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.081355] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319A-0-0] [lt=30] success to process renew task(task={tenant_id:1, table_id:11023, add_timestamp:1742176620081136}, locations=[{table_id:11023, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}, {table_id:11023, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620081313, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.081517] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC9F-0-0] [lt=22][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.081571] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFEC9F-0-0] [lt=23][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.081822] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=82][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f79906eadb0, session_id:3222503872, trace_id:YB420A0B093E-00062FB7F0DFECA1-0-0, sql_handling_stage:2, sql_initiative_shutdown:false, reader:{fd:301}, err:5, last_decode_time:1742176620081447, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2207, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.081863] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=35] kill and revert session(conn.sessid_=3222503872, proxy_sessid=0, ret=0) [2025-03-17 09:57:00.081892] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=9][errcode=0] server close connection(sessid=3222503872, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:00.081915] INFO [SERVER] process (obmp_quit.h:56) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=21] quit [2025-03-17 09:57:00.081924] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=8][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:57:00.081933] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=8][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:57:00.081940] WDIAG [SERVER] response (obmp_base.cpp:87) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=7][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:57:00.081948] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=7][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:57:00.081956] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.081965] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA1-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.081969] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118871][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=3][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.083850] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703445}], scn_array=[cnt:1, {val:1742176619656135004, v:0}], curr_size=122, accum_size=854, time_cost=24355) [2025-03-17 09:57:00.086580] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B85901-0-0] [lt=8][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.086650] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B85901-0-0] [lt=20][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.088203] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B85902-0-0] [lt=12] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12291, add_timestamp:1742176620088192}) [2025-03-17 09:57:00.088242] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319B-0-0] [lt=17] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12291, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620088229, sql_renew_time:1742176620088229, is_mark_fail:false}]) [2025-03-17 09:57:00.088269] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319B-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:12291, add_timestamp:1742176620088192}, locations=[{table_id:12291, partition_id:12291, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620088229, sql_renew_time:1742176620088229, is_mark_fail:false}]) [2025-03-17 09:57:00.096407] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=13] can close safely, do destroy(*s={this:0x7f79906eadb0, session_id:3222503872, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:301}, err:5, last_decode_time:1742176620081447, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2212, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:57:00.096507] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=62] unlock session of tenant(conn.sessid_=3222503872, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:57:00.096521] INFO [SERVER] run (obmp_disconnect.cpp:83) [117889][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=40] free session successfully(sessid=3222503872) [2025-03-17 09:57:00.096540] INFO [SERVER] run (obmp_disconnect.cpp:90) [117889][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=18] mark session id unused(sessid=3222503872) [2025-03-17 09:57:00.096526] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=18] connection close(sessid=3222503872, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB7F0DFEC9F-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:44, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:229, rec_:229, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[14]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_mysql_header_{pkt_num_:5, seq_:7}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_mysql_header_{len_:9, seq_:8}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{send:obp_mysql_header_{pkt_num_:1, seq_:9}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[18]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{receive:obp_mysql_header_{len_:362, rec_:362, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[20]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[21]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[22]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[23]:{send:obp_mysql_header_{pkt_num_:1, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[25]:{receive:obp_mysql_header_{len_:181, rec_:181, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[26]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{send:obp_mysql_header_{pkt_num_:2, seq_:4}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[28]:{send:obp_mysql_header_{len_:9, seq_:5}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_mysql_header_{pkt_num_:1, seq_:6}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[31]:{receive:obp_mysql_header_{len_:59, rec_:59, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[32]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[33]:{send:obp_mysql_header_{pkt_num_:661, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[34]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[35]:{send:obp_mysql_header_{pkt_num_:6, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[36]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[37]:{receive:obp_mysql_header_{len_:407, rec_:407, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[38]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[39]:{send:obp_mysql_header_{pkt_num_:183, seq_:5}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[40]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[41]:{send:obp_mysql_header_{pkt_num_:7, seq_:8}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[42]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[43]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:57:00.097397] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117623][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=61] new mysql sessid created(conn.sessid_=3222518581, support_ssl=false) [2025-03-17 09:57:00.097457] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117623][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=19] sm conn init succ(conn.sessid_=3222518581, sess.client_addr_="127.0.0.1:9368") [2025-03-17 09:57:00.097472] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117623][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=11] accept one succ(*s={this:0x7f7ad287cdb0, session_id:3222518581, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:251}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.097533] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117623][sql_nio4][T0][Y0-0000000000000000-0-0] [lt=14] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.097559] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117623][sql_nio4][T1][Y0-0000000000000000-0-0] [lt=10] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.097621] INFO load_privilege_info (obmp_connect.cpp:582) [117811][T1_MysqlQueueTh][T1][Y0-00062FB7F07684F2-0-0] [lt=38] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.098460] INFO [SERVER] process (obmp_connect.cpp:523) [117811][T1_MysqlQueueTh][T1][Y0-00062FB7F07684F2-0-0] [lt=21] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222518581, proxy_sessid=0, client_sessid=3222518581, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.102068] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA4-0-0] [lt=4][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.102481] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=97][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7ad287c030, session_id:3222489765, trace_id:YB420A0B093E-00062FB7F0DFECA5-0-0, sql_handling_stage:2, sql_initiative_shutdown:false, reader:{fd:300}, err:5, last_decode_time:1742176620101648, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2611, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.102512] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=26][errcode=0] server close connection(sessid=3222489765, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:00.102519] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=33] kill and revert session(conn.sessid_=3222489765, proxy_sessid=0, ret=0) [2025-03-17 09:57:00.102527] INFO [SERVER] process (obmp_quit.h:56) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=14] quit [2025-03-17 09:57:00.102535] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=7][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:57:00.102542] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=7][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:57:00.102555] WDIAG [SERVER] response (obmp_base.cpp:87) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=12][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:57:00.102563] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=8][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:57:00.102571] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.102580] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118871][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0DFECA5-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.102588] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118871][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.103689] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B85903-0-0] [lt=20] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12317, add_timestamp:1742176620103674}) [2025-03-17 09:57:00.103715] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319C-0-0] [lt=15] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12317, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.103898] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319C-0-0] [lt=20] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12317, locations=[{table_id:12317, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.103925] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319C-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:12317, add_timestamp:1742176620103674}, locations=[{table_id:12317, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}, {table_id:12317, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620103885, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.109906] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=27] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:14993, add_timestamp:1742176620109892}) [2025-03-17 09:57:00.109927] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319D-0-0] [lt=19] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=14993, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.110100] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319D-0-0] [lt=21] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=14993, locations=[{table_id:14993, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.110127] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319D-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:14993, add_timestamp:1742176620109892}, locations=[{table_id:14993, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}, {table_id:14993, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620110089, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.116463] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1003][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=29][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1003) [2025-03-17 09:57:00.116491] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1004][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=26][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1004) [2025-03-17 09:57:00.116497] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=5][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:57:00.116611] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=6][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1) [2025-03-17 09:57:00.116627] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1003][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=14][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1003) [2025-03-17 09:57:00.116632] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1004][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=4][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1004) [2025-03-17 09:57:00.116636] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=4][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:57:00.116665] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=4][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1) [2025-03-17 09:57:00.116683] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1003][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=8][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1003) [2025-03-17 09:57:00.118451] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_tablet_pointer_status.cpp:73) [118838][T1_L1_G10000][T1][YB420A0B093E-00062FB937B85903-0-0] [lt=23][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.118578] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=14] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:00.121106] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=4][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:57:00.121224] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=20][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1) [2025-03-17 09:57:00.121236] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1003][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=10][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1003) [2025-03-17 09:57:00.121242] WDIAG [SERVER] process_curr_tenant (ob_all_virtual_sys_stat.cpp:321) [118873][T1_L0_G10000][T1004][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=4][errcode=-4008] get diag info fail(ret=-4008, tenant_id_=1004) [2025-03-17 09:57:00.121246] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=3][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:57:00.121258] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=3][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.121267] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=8][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.123411] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=34] new mysql sessid created(conn.sessid_=3222511607, support_ssl=false) [2025-03-17 09:57:00.123467] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=33] sm conn init succ(conn.sessid_=3222511607, sess.client_addr_="127.0.0.1:9374") [2025-03-17 09:57:00.123481] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=10] accept one succ(*s={this:0x7f7af9aa8db0, session_id:3222511607, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:285}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.123528] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=17] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.123556] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117620][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=10] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.123639] INFO load_privilege_info (obmp_connect.cpp:582) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A1-0-0] [lt=29] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.123805] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:00.123850] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:6, fail_bw:24537, [delay/us]:prepare:1, schedule:23, submit:0, rt:358, total:384, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:00.123869] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4505, iops:10, bw:44979, [delay/us]:prepare:5, schedule:21, submit:404, rt:15603, total:16035) [2025-03-17 09:57:00.123879] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=2, iops=2, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:00.123912] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:6, avg_bw:24537, [delay/us]: prepare:1, schedule:23, submit:0, device:358, total:384) [2025-03-17 09:57:00.123928] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4506, avg_iops:10, avg_bw:44985, [delay/us]: prepare:5, schedule:21, submit:404, device:15603, total:16035) [2025-03-17 09:57:00.123938] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:57:00.123967] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:12, fail_bw:49074, [delay/us]:prepare:1, schedule:28, submit:1, rt:392, total:424, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:00.123981] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4726, iops:13, bw:61343, [delay/us]:prepare:8, schedule:23, submit:23, rt:23696, total:23752) [2025-03-17 09:57:00.123989] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:00.124007] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:12, avg_bw:49076, [delay/us]: prepare:1, schedule:28, submit:1, device:392, total:424) [2025-03-17 09:57:00.124021] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4726, avg_iops:13, avg_bw:61345, [delay/us]: prepare:8, schedule:23, submit:23, device:23696, total:23752) [2025-03-17 09:57:00.124031] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:57:00.124060] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:20, bw:81794, [delay/us]:prepare:5, schedule:19, submit:365, rt:17688, total:18078) [2025-03-17 09:57:00.124068] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1004, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33434400, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:00.124093] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4096, avg_iops:20, avg_bw:81796, [delay/us]: prepare:5, schedule:19, submit:365, device:17688, total:18078) [2025-03-17 09:57:00.124103] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:57:00.124122] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4089, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:465, total:473) [2025-03-17 09:57:00.124154] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:00.124171] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=15] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4089, [delay/us]: prepare:0, schedule:0, submit:600000000, device:465, total:473) [2025-03-17 09:57:00.124186] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:57:00.124197] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=1, sync_total=0, async_cnt= 1 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:00.124206] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124214] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124220] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124226] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124230] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124236] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124240] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124248] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:00.124258] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124268] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124272] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124278] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:00.124289] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SERVER] net_in=338kB/s, net_out=1602kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:57:00.124280] INFO [SERVER] process (obmp_connect.cpp:523) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A1-0-0] [lt=28] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222511607, proxy_sessid=0, client_sessid=3222511607, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.124785] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACAFF-0-0] [lt=34] get number(ret=0, service_type_=0, range=1, base_id=1742176620124156000, start_id=1742176620124156000, end_id=1742176620124156001) [2025-03-17 09:57:00.125610] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=37] new mysql sessid created(conn.sessid_=3222520260, support_ssl=false) [2025-03-17 09:57:00.125658] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=22] sm conn init succ(conn.sessid_=3222520260, sess.client_addr_="127.0.0.1:9388") [2025-03-17 09:57:00.125672] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=11] accept one succ(*s={this:0x7f7af9b46db0, session_id:3222520260, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:301}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.125728] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=25] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.125753] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117624][sql_nio5][T1][Y0-0000000000000000-0-0] [lt=10] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.125812] INFO load_privilege_info (obmp_connect.cpp:582) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED68360-0-0] [lt=29] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.126525] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118838][T1_L1_G10000][T1][YB420A0B093E-00062FB8C01ACAFF-0-0] [lt=26] get_stable_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:57:00.126571] INFO [SERVER] process (obmp_connect.cpp:523) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED68360-0-0] [lt=27] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222520260, proxy_sessid=0, client_sessid=3222520260, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.126605] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118838][T1_L1_G10000][T1003][YB420A0B093E-00062FB8C01ACAFF-0-0] [lt=24] get_stable_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:57:00.126626] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118838][T1_L1_G10000][T1004][YB420A0B093E-00062FB8C01ACAFF-0-0] [lt=9] get_stable_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:57:00.127311] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118860][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5715-0-0] [lt=28] get_stable_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:57:00.127361] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118860][T1_L0_G10000][T1003][YB420A0B093E-00062FB7F06F5715-0-0] [lt=23] get_stable_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:57:00.127382] INFO [PALF] get_stable_disk_usage (palf_env_impl.cpp:934) [118860][T1_L0_G10000][T1004][YB420A0B093E-00062FB7F06F5715-0-0] [lt=11] get_stable_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:57:00.127732] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=35] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:57:00.127761] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=20] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=2, times=11821) [2025-03-17 09:57:00.129384] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=52] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=22877184) [2025-03-17 09:57:00.129441] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=6] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:00.130108] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118860][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5715-0-0] [lt=11][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.130680] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5716-0-0] [lt=15] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:11001, add_timestamp:1742176620130665}) [2025-03-17 09:57:00.130706] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319E-0-0] [lt=35] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=11001, vtable_type={tenant_id:1, type:2}) [2025-03-17 09:57:00.130855] WDIAG [SERVER] inner_get_next_row (ob_all_virtual_sys_stat.cpp:301) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCA-0-0] [lt=6][errcode=-4008] execute fail(ret=-4008) [2025-03-17 09:57:00.130912] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319E-0-0] [lt=21] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=11001, locations=[{table_id:11001, partition_id:0, partition_cnt:1, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620130897, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.130940] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319E-0-0] [lt=28] success to process renew task(task={tenant_id:1, table_id:11001, add_timestamp:1742176620130665}, locations=[{table_id:11001, partition_id:0, partition_cnt:1, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620130897, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.133555] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACAFF-0-0] [lt=25][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.134002] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=18][errcode=0] server close connection(sessid=3222511607, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:00.134023] INFO [SERVER] process (obmp_quit.h:56) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=20] quit [2025-03-17 09:57:00.134031] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=7][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:57:00.134012] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9aa8db0, session_id:3222511607, trace_id:YB420A0B093E-00062FB8C01ACB00-0-0, sql_handling_stage:10, sql_initiative_shutdown:true, reader:{fd:285}, err:5, last_decode_time:1742176620133733, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:775, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.134041] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=10][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:57:00.134047] WDIAG [SERVER] response (obmp_base.cpp:87) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=6][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:57:00.134049] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=31] kill and revert session(conn.sessid_=3222511607, proxy_sessid=0, ret=0) [2025-03-17 09:57:00.134053] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=6][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:57:00.134061] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.134059] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] can close safely, do destroy(*s={this:0x7f7af9aa8db0, session_id:3222511607, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:285}, err:5, last_decode_time:1742176620133733, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:780, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:57:00.134070] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB00-0-0] [lt=8][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.134080] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] unlock session of tenant(conn.sessid_=3222511607, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:57:00.134081] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118870][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=10][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.134087] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=6] connection close(sessid=3222511607, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB8C01ACAFF-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:8, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:670, rec_:670, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:8, seq_:17}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:18}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:57:00.134121] INFO [SERVER] run (obmp_disconnect.cpp:83) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=18] free session successfully(sessid=3222511607) [2025-03-17 09:57:00.134140] INFO [SERVER] run (obmp_disconnect.cpp:90) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=18] mark session id unused(sessid=3222511607) [2025-03-17 09:57:00.134227] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859700618}], scn_array=[cnt:1, {val:1742176619995225001, v:0}], curr_size=122, accum_size=4389, time_cost=24187) [2025-03-17 09:57:00.134249] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=32] Slow write(duration=42408, write_len=4096) [2025-03-17 09:57:00.134268] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107783, offset=57287103}, end_cursor:ObLogCursor{file_id=5, log_id=1107784, offset=57287409}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17704050, len:3649, log_data_len:306, seq:1107783, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:00.134415] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:57:00.134463] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=33] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176619656135003, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176619656135003, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:00.136364] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=11][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.136791] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=31] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=7450) [2025-03-17 09:57:00.143013] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=9][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9b46db0, session_id:3222520260, trace_id:YB420A0B093E-00062FB7F06F5717-0-0, sql_handling_stage:2, sql_initiative_shutdown:false, reader:{fd:301}, err:5, last_decode_time:1742176620142256, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:687, pending_flag:1, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.143043] WDIAG [SERVER] disconnect (obmp_packet_sender.cpp:869) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=26][errcode=0] server close connection(sessid=3222520260, proxy_sessid=0, stack="0x810107f 0x101960e0 0x10164efb 0x10315ac1 0x78b1ae0 0x78aefed 0x789e119 0xfc77119 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:00.143054] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=36] kill and revert session(conn.sessid_=3222520260, proxy_sessid=0, ret=0) [2025-03-17 09:57:00.143058] INFO [SERVER] process (obmp_quit.h:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=14] quit [2025-03-17 09:57:00.143065] WDIAG [RPC.OBMYSQL] async_write_data (ob_sql_sock_session.cpp:163) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=6][errcode=-4009] sock has error(ret=-4009) [2025-03-17 09:57:00.143071] WDIAG [SERVER] flush_buffer (obmp_packet_sender.cpp:1072) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=6][errcode=-4009] write response fail(ret=-4009) [2025-03-17 09:57:00.143076] WDIAG [SERVER] response (obmp_base.cpp:87) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=4][errcode=-4009] failed to flush_buffer(ret=-4009) [2025-03-17 09:57:00.143084] WDIAG [RPC.FRAME] run (ob_sql_processor.cpp:50) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=7][errcode=-4009] response rpc result fail(ret=-4009, tmp_ret=-4009) [2025-03-17 09:57:00.143092] WDIAG [SERVER.OMT] process_one (ob_worker_processor.cpp:89) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=8][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.143100] WDIAG [SERVER.OMT] process (ob_worker_processor.cpp:157) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5717-0-0] [lt=6][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.143106] WDIAG [SERVER.OMT] process_request (ob_th_worker.cpp:253) [118874][T1_L0_G10000][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4009] process request fail(ret=-4009) [2025-03-17 09:57:00.153407] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:449) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=19] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:00.153430] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=23] try reload schema success(schema_version=1741661297536408, refresh_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_ret=0) [2025-03-17 09:57:00.153439] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:263) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A1-0-0] [lt=14] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:00.153461] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:288) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A1-0-0] [lt=20] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:00.154438] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=14] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12084, add_timestamp:1742176620154415}) [2025-03-17 09:57:00.154471] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319F-0-0] [lt=21] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12084, locations=[{table_id:12084, partition_id:12084, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620154458, sql_renew_time:1742176620154458, is_mark_fail:false}]) [2025-03-17 09:57:00.154497] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E319F-0-0] [lt=26] success to process renew task(task={tenant_id:1, table_id:12084, add_timestamp:1742176620154415}, locations=[{table_id:12084, partition_id:12084, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620154458, sql_renew_time:1742176620154458, is_mark_fail:false}]) [2025-03-17 09:57:00.157553] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1742176620157204000, start_id=1742176620157204000, end_id=1742176620157204001) [2025-03-17 09:57:00.157600] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=32] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1742176620157545], gts:1742176620157204000, latest_srr:[mts=1742176620157545]}) [2025-03-17 09:57:00.157619] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=17] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:57:00.165223] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859700740}], scn_array=[cnt:1, {val:1742176620131603000, v:0}], curr_size=588, accum_size=4977, time_cost=30914) [2025-03-17 09:57:00.165271] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463513771}], scn_array=[cnt:1, {val:1742176620146518001, v:0}], curr_size=586, accum_size=1173, time_cost=18183) [2025-03-17 09:57:00.165494] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=29] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:57:00.173069] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703567}], scn_array=[cnt:1, {val:1742176620157204000, v:0}], curr_size=122, accum_size=976, time_cost=13564) [2025-03-17 09:57:00.173179] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415712}], scn_array=[cnt:1, {val:1742176620157204000, v:0}], curr_size=122, accum_size=1098, time_cost=13645) [2025-03-17 09:57:00.173890] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:559) [117778][T1_HBService][T1][YB420A0B093E-00062FB7F27BCA1A-0-0] [lt=15] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2025-03-17 09:57:00.174474] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=33] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:57:00.180913] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.180909] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=23][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.180937] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=27][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.180961] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=14][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.180968] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.180982] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.180997] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=14][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181008] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181020] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181033] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181042] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181056] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181064] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181074] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181092] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=18][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181103] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181111] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181125] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181134] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181176] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=38][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181182] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=5][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181191] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181203] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181216] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181229] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181242] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181252] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181266] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181275] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181288] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181300] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181314] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181320] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181335] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181353] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=18][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181370] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181380] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181396] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181406] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181434] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181439] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=5][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181450] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181460] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181475] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181486] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181521] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=31][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181531] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181545] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181553] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181567] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181576] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181602] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=22][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181611] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181625] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181634] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181648] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181657] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181672] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181680] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181694] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181704] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181714] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181723] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181737] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181746] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181760] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181770] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181784] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181793] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181807] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181817] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181832] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181842] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181855] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181865] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181878] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181871] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=32] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:57:00.181888] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181901] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181910] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181924] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181933] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181947] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181957] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=10][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181973] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=12][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.181983] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.181997] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=11][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182007] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182019] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=9][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182025] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182031] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=4][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182038] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182048] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182054] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182064] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182070] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182080] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182054] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=19] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107784, offset=57287409}, end_cursor:ObLogCursor{file_id=5, log_id=1107785, offset=57287706}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b1771c050, len:3343, log_data_len:297, seq:1107784, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:00.182086] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182095] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182102] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182111] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182117] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182126] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182130] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=4][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182139] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182158] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=19][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182166] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=5][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182169] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=4][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182184] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=13][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182190] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=6][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182200] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182207] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182216] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] char type length is too long(max_accuracy_len=8, trunc_len_char=11) [2025-03-17 09:57:00.182223] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12277) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=7][errcode=0] string accuracy check failed(cast_ret=-5167, ret=0, warning=-5167, is_oracle=false, meta={type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}) [2025-03-17 09:57:00.182233] WDIAG [SQL] string_length_check (ob_datum_cast.cpp:12264) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCB-0-0] [lt=8][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.182246] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=28] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:57:00.182282] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731B-0-0] [lt=22] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176620157204000, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176620157204000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:00.183531] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539162][T1004_Occam][T1004][YB420A0B093E-00062FB7FA8C57C0-0-0] [lt=17] for each ls(succ_num=2, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:00.184887] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463514357}], scn_array=[cnt:1, {val:1742176620163596000, v:0}], curr_size=1176, accum_size=2349, time_cost=19515) [2025-03-17 09:57:00.194923] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC6-0-0] [lt=22] [RS_MGR] force set rs list(master_rs="192.168.1.62:2882") [2025-03-17 09:57:00.194986] INFO [SERVER] handle_heartbeat (ob_service.cpp:3753) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC6-0-0] [lt=58] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"192.168.1.62:2882", server_id:4, rs_addr:"192.168.1.62:2882", rs_server_status:1, epoch_id:4}, hb_response={zone:"zone1", server:"192.168.1.62:2882", sql_port:2881, build_version:"4.3.5.0_100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5(Dec 31 2024 17:35:01)", start_service_time:1741316885024955, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=64) [2025-03-17 09:57:00.197666] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=9] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40298826956, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:00.198778] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=30] PNIO [ratelimit] time: 1742176620198775, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007611, add_bytes: 0 [2025-03-17 09:57:00.198799] INFO pn_print_diag_info (group.c:775) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=20] PNIO client_send:0/0, client_queue_time=0, cnt=0, server_send:0/0, server_queue_time=0, cnt=0 [2025-03-17 09:57:00.205499] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:671) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=34] Cache wash clean map node details(ret=0, clean_node_count=0, clean_time=0, clean_start_pos=5800000, clean_num=200000) [2025-03-17 09:57:00.215355] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175477][T1_L0_G5][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=9] get number(ret=0, service_type_=0, range=1, base_id=1742176620214519000, start_id=1742176620214519000, end_id=1742176620214519001) [2025-03-17 09:57:00.216296] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117899][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=6][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.216319] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117899][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.216333] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117899][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f786ec0c690, local_datums:0x7f786ec0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f786ec0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f786ec0c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c36e, hex: C874951A80300600, int: 1742176620213448}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c376, hex: C874951A80300600, int: 1742176620213448}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f786ec0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.216442] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117899][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=42][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.216711] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=35] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2582}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397090} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.216765] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:678) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90A-0-0] [lt=34] rpc statistics(total_trans_msg_count=73, total_batch_msg_count=0) [2025-03-17 09:57:00.218543] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90B-0-0] [lt=11] get number(ret=0, service_type_=0, range=1, base_id=1742176620217700000, start_id=1742176620217700000, end_id=1742176620217700001) [2025-03-17 09:57:00.229945] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:257) [3539257][T1004_ArcSender][T1004][YB420A0B093E-00062FB7F8454125-0-0] [lt=35][errcode=-4018] try free send task failed(ret=-4018) [2025-03-17 09:57:00.235607] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859703209}], scn_array=[cnt:1, {val:1742176620195447000, v:0}], curr_size=588, accum_size=7446, time_cost=39744) [2025-03-17 09:57:00.236264] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463515533}], scn_array=[cnt:1, {val:1742176620217700000, v:0}], curr_size=551, accum_size=2900, time_cost=17621) [2025-03-17 09:57:00.238472] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90C-0-0] [lt=20][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.238494] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90C-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.238504] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90C-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c6a0, local_datums:0x7f782b80c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c370, hex: D8CE951A80300600, int: 1742176620236504}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c378, hex: D8CE951A80300600, int: 1742176620236504}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.238604] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90C-0-0] [lt=48][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.238946] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90C-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1067}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397091} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.247190] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:00.247240] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=50][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:00.247341] INFO pn_print_diag_info (group.c:755) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO client:0x7f7b1edfe048_"192.168.1.64:2882"_"192.168.1.64:2882"_292_1741767244436101_1, write_queue=0/0, write=1010018/1099534764, read=1010017/70234794814, doing=0, done=1010018, write_time=1034638018, read_time=26543260, process_time=26625868 [2025-03-17 09:57:00.247358] INFO pn_print_diag_info (group.c:755) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=15] PNIO client:0x7f7af01eb4a8_"192.168.1.65:2882"_"192.168.1.65:2882"_280_1741767244231536_1, write_queue=0/0, write=862413/1014828205, read=862413/69205991561, doing=0, done=862413, write_time=881536503, read_time=25760052, process_time=25824137 [2025-03-17 09:57:00.247365] INFO pn_print_diag_info (group.c:755) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x7f7b1edf2a58_"192.168.1.62:2882"_"192.168.1.62:2882"_179_1741316879933718_1, write_queue=0/0, write=2497198/2925925305, read=2497196/619294535, doing=0, done=2497198, write_time=2552195906, read_time=195837, process_time=381067 [2025-03-17 09:57:00.247371] INFO pn_print_diag_info (group.c:755) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x7f7b1ecfa048_"192.168.1.63:2882"_"192.168.1.63:2882"_190_1741316878938585_1, write_queue=1/338, write=2138343/2354132272, read=2138343/158300859393, doing=1, done=2138343, write_time=2187768016, read_time=53569162, process_time=53746279 [2025-03-17 09:57:00.247376] INFO pn_print_diag_info (group.c:768) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7af01fea58_"192.168.1.64:20986"_309_1741767246060436, write_queue=0/0, write=2674916/1095264581, read=2674916/3552995637, doing=0, done=2674916, write_time=2728019494, read_time=1991867, process_time=2148142 [2025-03-17 09:57:00.247382] INFO pn_print_diag_info (group.c:768) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7b1ecfb4e8_"192.168.1.65:25826"_248_1741767244187110, write_queue=0/0, write=2616935/1068319299, read=2616935/3385031713, doing=0, done=2616935, write_time=2668867714, read_time=2097160, process_time=2246315 [2025-03-17 09:57:00.247393] INFO pn_print_diag_info (group.c:768) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=11] PNIO server:0x7f7b1edff4e8_"192.168.1.63:41042"_208_1741316880512050, write_queue=0/0, write=5398710/3119641186, read=5398710/6943965067, doing=0, done=5398710, write_time=5503130167, read_time=4183678, process_time=4486854 [2025-03-17 09:57:00.247399] INFO pn_print_diag_info (group.c:768) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7b1edf4a98_"192.168.1.62:16200"_181_1741316879934568, write_queue=0/0, write=2497196/619294535, read=2497198/2925925305, doing=0, done=2497198, write_time=2537073193, read_time=313292, process_time=455906 [2025-03-17 09:57:00.247403] INFO pn_print_diag_info (group.c:775) [117492][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client_send:7720739/9250965747, client_queue_time=162351302, cnt=4, server_send:18146865/7816869796, server_queue_time=334355879, cnt=4 [2025-03-17 09:57:00.247692] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.248608] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFB0-0-0] [lt=8][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.248649] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:00.248657] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:57:00.252332] INFO pn_print_diag_info (group.c:755) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7ae97eb4e8_"192.168.1.64:2882"_"192.168.1.64:2882"_286_1741767244425275_1, write_queue=0/0, write=1039180/1127966057, read=1039180/72292995094, doing=0, done=1039180, write_time=1063727788, read_time=26950190, process_time=27015622 [2025-03-17 09:57:00.252361] INFO pn_print_diag_info (group.c:755) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=28] PNIO client:0x7f7ae97614a8_"192.168.1.65:2882"_"192.168.1.65:2882"_232_1741767244166312_1, write_queue=0/0, write=884880/1037917744, read=884880/71124492104, doing=0, done=884880, write_time=904130545, read_time=26325893, process_time=26389981 [2025-03-17 09:57:00.252369] INFO pn_print_diag_info (group.c:755) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO client:0x7f7ae9604048_"192.168.1.62:2882"_"192.168.1.62:2882"_215_1741316881936789_1, write_queue=0/0, write=2566504/3008756154, read=2566504/636526124, doing=0, done=2566504, write_time=2623318038, read_time=202076, process_time=392126 [2025-03-17 09:57:00.252376] INFO pn_print_diag_info (group.c:755) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x7f7b1ec6e048_"192.168.1.63:2882"_"192.168.1.63:2882"_187_1741316878933428_1, write_queue=0/0, write=2197804/2407038489, read=2197804/162651565892, doing=0, done=2197804, write_time=2249325627, read_time=54754323, process_time=54934026 [2025-03-17 09:57:00.252381] INFO pn_print_diag_info (group.c:768) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7ae97fca98_"192.168.1.64:20948"_298_1741767246047567, write_queue=0/0, write=2750960/1129590391, read=2750960/3668557304, doing=0, done=2750960, write_time=2807423921, read_time=2065194, process_time=2222842 [2025-03-17 09:57:00.252390] INFO pn_print_diag_info (group.c:768) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO server:0x7f7ae968ea58_"192.168.1.65:25798"_235_1741767244184310, write_queue=0/0, write=2689150/1086576097, read=2689150/3472784868, doing=0, done=2689150, write_time=2741956616, read_time=2071816, process_time=2227768 [2025-03-17 09:57:00.252400] INFO pn_print_diag_info (group.c:768) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO server:0x7f7ae97ec048_"192.168.1.63:12014"_233_1741316885617256, write_queue=0/0, write=5549913/3218251287, read=5549913/7141985660, doing=0, done=5549913, write_time=5656146213, read_time=4235697, process_time=4550095 [2025-03-17 09:57:00.252414] INFO pn_print_diag_info (group.c:768) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO server:0x7f7ae9604a98_"192.168.1.62:16234"_216_1741316881938098, write_queue=0/0, write=2566504/636526124, read=2566504/3008756154, doing=0, done=2566504, write_time=2608548995, read_time=323852, process_time=470091 [2025-03-17 09:57:00.252427] INFO pn_print_diag_info (group.c:775) [117489][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO client_send:7935234/9477732580, client_queue_time=166373205, cnt=4, server_send:18654987/8046526480, server_queue_time=343301913, cnt=4 [2025-03-17 09:57:00.253704] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=25] ====== check clog disk timer task ====== [2025-03-17 09:57:00.253731] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=24] get_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:57:00.256869] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=31] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=8) [2025-03-17 09:57:00.274038] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12070, add_timestamp:1742176620274020}) [2025-03-17 09:57:00.274098] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=39] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12076, add_timestamp:1742176620274096}) [2025-03-17 09:57:00.274114] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=11] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12082, add_timestamp:1742176620274113}) [2025-03-17 09:57:00.274090] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A0-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12070, locations=[{table_id:12070, partition_id:12070, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274071, sql_renew_time:1742176620274071, is_mark_fail:false}]) [2025-03-17 09:57:00.274128] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=9] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12241, add_timestamp:1742176620274128}) [2025-03-17 09:57:00.274127] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A0-0-0] [lt=35] success to process renew task(task={tenant_id:1, table_id:12070, add_timestamp:1742176620274020}, locations=[{table_id:12070, partition_id:12070, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274071, sql_renew_time:1742176620274071, is_mark_fail:false}]) [2025-03-17 09:57:00.274162] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A1-0-0] [lt=14] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12076, locations=[{table_id:12076, partition_id:12076, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274157, sql_renew_time:1742176620274157, is_mark_fail:false}]) [2025-03-17 09:57:00.274175] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A1-0-0] [lt=13] success to process renew task(task={tenant_id:1, table_id:12076, add_timestamp:1742176620274096}, locations=[{table_id:12076, partition_id:12076, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274157, sql_renew_time:1742176620274157, is_mark_fail:false}]) [2025-03-17 09:57:00.274194] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A2-0-0] [lt=12] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12082, locations=[{table_id:12082, partition_id:12082, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274189, sql_renew_time:1742176620274189, is_mark_fail:false}]) [2025-03-17 09:57:00.274212] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A2-0-0] [lt=18] success to process renew task(task={tenant_id:1, table_id:12082, add_timestamp:1742176620274113}, locations=[{table_id:12082, partition_id:12082, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274189, sql_renew_time:1742176620274189, is_mark_fail:false}]) [2025-03-17 09:57:00.274235] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A3-0-0] [lt=14] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12241, locations=[{table_id:12241, partition_id:12241, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274230, sql_renew_time:1742176620274230, is_mark_fail:false}]) [2025-03-17 09:57:00.274251] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A3-0-0] [lt=16] success to process renew task(task={tenant_id:1, table_id:12241, add_timestamp:1742176620274128}, locations=[{table_id:12241, partition_id:12241, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620274230, sql_renew_time:1742176620274230, is_mark_fail:false}]) [2025-03-17 09:57:00.280991] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.281585] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17641, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:57:00.281614] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162415834}], scn_array=[cnt:1, {val:1742176620157204001, v:0}], curr_size=122, accum_size=1220, time_cost=21972) [2025-03-17 09:57:00.281637] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:57:00.281632] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463516084}], scn_array=[cnt:1, {val:1742176620240104000, v:0}], curr_size=555, accum_size=3455, time_cost=40810) [2025-03-17 09:57:00.281656] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=25, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:57:00.281658] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703689}], scn_array=[cnt:1, {val:1742176620157204001, v:0}], curr_size=122, accum_size=1098, time_cost=22063) [2025-03-17 09:57:00.281674] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:57:00.281706] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:57:00.281722] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17747, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:57:00.281736] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=38, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:57:00.281731] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78624-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c6498, ref_cnt_:1, raw_buf_:0x7f7aaac32170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78624-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176620267020, enqueue_used:2, dequeue_used:27, submit_used:1, return_used:14641, callback_enqueue_used:3, callback_dequeue_used:35, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:00.281802] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78624-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.281838] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78624-0-0] [lt=7][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.282256] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78624-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade9a1620, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519792, assoc_session_id:3222519792, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620266804, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620265743, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a393db118, id:3222519792, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade9a1620}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.283522] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539350][T1003_Occam][T1003][YB420A0B093E-00062FB7FA9C57C0-0-0] [lt=28] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:00.283936] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90E-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.283962] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90E-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.283970] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90E-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36d, hex: 6581961A80300600, int: 1742176620282213}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c375, hex: 6581961A80300600, int: 1742176620282213}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.284064] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF90E-0-0] [lt=44][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.284486] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117878][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF90E-0-0] [lt=11] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2744}, to_seq_no={branch:0, seq:2146}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397092} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.295261] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=32] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:723, free_obj_hold_cnt:78, allocator used:3082248, allocator total:3329280}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:233, free_obj_hold_cnt:724, allocator used:62770587, allocator total:62770587}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():130944, full allocator used:1600, full allocator total:196480}, tablets_mem=66296347, tablets_mem_limit=3092376440, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:88, allocator used:231424, allocator total:261120}, pending_cnt=0, wait_gc_count=0, tablet count=956) [2025-03-17 09:57:00.304751] INFO [LIB] log_compress_loop_ (ob_log_compressor.cpp:411) [117414][SyslogCompress][T0][Y0-0000000000000000-0-0] [lt=28] log compressor cycles once. (ret=0, cost_time=1058, compressed_file_count=0, deleted_file_count=0, disk_remaining_size=17882951680, fast_delete_log_mode=false) [2025-03-17 09:57:00.306908] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=40] PNIO [ratelimit] time: 1742176620306907, bytes: 2535008257516, bw: 2.799825 MB/s, add_ts: 1000015, add_bytes: 2935873 [2025-03-17 09:57:00.306929] INFO pn_print_diag_info (group.c:755) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=18] PNIO client:0x7f7ae97fc048_"192.168.1.65:2882"_"192.168.1.65:2882"_295_1741767245224897_1, write_queue=0/0, write=885925/1048657820, read=885925/71106125855, doing=0, done=885925, write_time=905800883, read_time=25683368, process_time=25754869 [2025-03-17 09:57:00.306938] INFO pn_print_diag_info (group.c:755) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7ae97eaa98_"192.168.1.64:2882"_"192.168.1.64:2882"_283_1741767244411789_1, write_queue=0/0, write=1039704/1137215873, read=1039704/72202399928, doing=0, done=1039704, write_time=1063842845, read_time=26488455, process_time=26574369 [2025-03-17 09:57:00.306947] INFO pn_print_diag_info (group.c:755) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7ae96054a8_"192.168.1.62:2882"_"192.168.1.62:2882"_218_1741316881950508_1, write_queue=0/0, write=2565110/2996694929, read=2565109/636230189, doing=0, done=2565110, write_time=2623105547, read_time=201171, process_time=387431 [2025-03-17 09:57:00.306956] INFO pn_print_diag_info (group.c:755) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7b1ec04808_"192.168.1.63:2882"_"192.168.1.63:2882"_182_1741316878926990_1, write_queue=0/0, write=2196134/2407191839, read=2196134/162605090663, doing=0, done=2196134, write_time=2248956809, read_time=53422530, process_time=53600032 [2025-03-17 09:57:00.306961] INFO pn_print_diag_info (group.c:768) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7ae97fea58_"192.168.1.64:21004"_311_1741767246068357, write_queue=0/0, write=2749821/1130884320, read=2749821/3659349323, doing=0, done=2749821, write_time=2805244212, read_time=2071087, process_time=2231174 [2025-03-17 09:57:00.306967] INFO pn_print_diag_info (group.c:768) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7ae4dfea98_"192.168.1.65:25856"_293_1741767244551274, write_queue=1/318, write=2687147/1090813041, read=2687148/3474729935, doing=1, done=2687147, write_time=2740760369, read_time=2182932, process_time=2336927 [2025-03-17 09:57:00.306972] INFO pn_print_diag_info (group.c:768) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7ae968e048_"192.168.1.62:16248"_219_1741316881950716, write_queue=0/0, write=2565109/636230189, read=2565110/2996694929, doing=0, done=2565110, write_time=2606835627, read_time=321454, process_time=468268 [2025-03-17 09:57:00.306977] INFO pn_print_diag_info (group.c:768) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7af01c6a98_"192.168.1.63:41044"_210_1741316880613518, write_queue=0/0, write=5547339/3218204100, read=5547339/7147539489, doing=0, done=5547339, write_time=5654187583, read_time=4373832, process_time=4689183 [2025-03-17 09:57:00.306981] INFO pn_print_diag_info (group.c:775) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client_send:7934988/9492251713, client_queue_time=167170786, cnt=4, server_send:18647828/8068130442, server_queue_time=344388056, cnt=4 [2025-03-17 09:57:00.310137] INFO [LIB] stat (utility.h:1210) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:57:00.319069] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463516639}], scn_array=[cnt:1, {val:1742176620286037000, v:0}], curr_size=551, accum_size=4006, time_cost=32136) [2025-03-17 09:57:00.319181] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=1540, avg_log_gen_to_submit_time=1554, avg_log_submit_to_flush_time=20344, avg_log_submit_to_slide_time=20363) [2025-03-17 09:57:00.319243] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=6261, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:00.320798] INFO pn_print_diag_info (group.c:755) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x7f7ae4dff4a8_"192.168.1.65:2882"_"192.168.1.65:2882"_294_1741767244806061_1, write_queue=0/0, write=859682/1014878601, read=859682/68976678896, doing=0, done=859682, write_time=878833180, read_time=25617862, process_time=25680461 [2025-03-17 09:57:00.320827] INFO pn_print_diag_info (group.c:755) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=27] PNIO client:0x7f7ae4dfe048_"192.168.1.64:2882"_"192.168.1.64:2882"_289_1741767244431365_1, write_queue=0/0, write=1011635/1099091563, read=1011635/70307434162, doing=0, done=1011635, write_time=1036392874, read_time=26590541, process_time=26674122 [2025-03-17 09:57:00.320840] INFO pn_print_diag_info (group.c:755) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO client:0x7f7ae968f468_"192.168.1.62:2882"_"192.168.1.62:2882"_224_1741316882938515_1, write_queue=0/0, write=2498654/2925019597, read=2498654/619779852, doing=0, done=2498654, write_time=2554554044, read_time=194135, process_time=350040 [2025-03-17 09:57:00.320850] INFO pn_print_diag_info (group.c:755) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7b1ec6f4e8_"192.168.1.63:2882"_"192.168.1.63:2882"_189_1741316878935926_1, write_queue=0/0, write=2138214/2354336018, read=2138214/158374299186, doing=0, done=2138214, write_time=2189474188, read_time=53599073, process_time=53733972 [2025-03-17 09:57:00.320867] INFO pn_print_diag_info (group.c:768) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=16] PNIO server:0x7f7ae97fd4a8_"192.168.1.64:20974"_308_1741767246052058, write_queue=0/0, write=2675130/1103368778, read=2675130/3555457854, doing=0, done=2675130, write_time=2728632692, read_time=2012500, process_time=2166207 [2025-03-17 09:57:00.320881] INFO pn_print_diag_info (group.c:768) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=13] PNIO server:0x7f7ae97eea58_"192.168.1.65:25814"_241_1741767244186569, write_queue=0/0, write=2617870/1061222816, read=2617870/3385680032, doing=0, done=2617870, write_time=2670553560, read_time=2103122, process_time=2256529 [2025-03-17 09:57:00.320889] INFO pn_print_diag_info (group.c:768) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO server:0x7f7ae97ee048_"192.168.1.63:12020"_237_1741316887618788, write_queue=0/0, write=5398380/3119463525, read=5398380/6942562955, doing=0, done=5398380, write_time=5502478935, read_time=4134205, process_time=4443262 [2025-03-17 09:57:00.320898] INFO pn_print_diag_info (group.c:768) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO server:0x7f7ae9760a98_"192.168.1.62:16258"_226_1741316882939217, write_queue=0/0, write=2498654/619779852, read=2498654/2925019597, doing=0, done=2498654, write_time=2539304257, read_time=313336, process_time=456258 [2025-03-17 09:57:00.320908] INFO pn_print_diag_info (group.c:775) [117491][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO client_send:7720790/9239588463, client_queue_time=161974324, cnt=4, server_send:18151829/7814019937, server_queue_time=333952403, cnt=4 [2025-03-17 09:57:00.321552] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF910-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.321578] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF910-0-0] [lt=23][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.321588] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF910-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36d, hex: F013971A80300600, int: 1742176620319728}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c375, hex: F013971A80300600, int: 1742176620319728}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.321699] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF910-0-0] [lt=43][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.322024] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF910-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1434}, to_seq_no={branch:0, seq:1061}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397093} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.323108] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7A-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:57:00.323129] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7A-0-0] [lt=20] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:57:00.324444] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859703797}], scn_array=[cnt:1, {val:1742176620195447001, v:0}], curr_size=122, accum_size=7568, time_cost=14245) [2025-03-17 09:57:00.344901] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:00.347896] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=24] gc stale ls task succ [2025-03-17 09:57:00.359606] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] tx gc loop thread is running(MTL_ID()=1004) [2025-03-17 09:57:00.359634] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=27] try gc retain ctx [2025-03-17 09:57:00.359641] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] try to retry start_working [2025-03-17 09:57:00.359666] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] finish update min start scn(max_decided_scn={val:1742176620157204000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176610144250002, v:0}, keep_alive_scn_:{val:1742176615150772002, v:0}, update_ts_:1742176620358628}) [2025-03-17 09:57:00.360398] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176617654369002, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176627787617000, latest_log_ts:{val:1742176611145419001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:57:00.360541] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=89] finish update min start scn(max_decided_scn={val:1742176620157204000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176610144250002, v:0}, keep_alive_scn_:{val:1742176615150772002, v:0}, update_ts_:1742176620359804}) [2025-03-17 09:57:00.361325] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1004, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148081676232000, v:0}, clog_base_lsn:{lsn:4093390848}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176617654369002, v:0}, tablet_change_checkpoint_scn:{val:1742008506458625001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:57:00.365893] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463517190}], scn_array=[cnt:1, {val:1742176620323280000, v:0}], curr_size=551, accum_size=4557, time_cost=42339) [2025-03-17 09:57:00.365989] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=29] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=6812, avg_group_log_size=486) [2025-03-17 09:57:00.366023] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.366078] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=54][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.366099] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:00.366109] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8898, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63401-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620346213, enqueue_used:2, dequeue_used:31, submit_used:1, return_used:19745, callback_enqueue_used:4, callback_dequeue_used:24, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.366165] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=48][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8898, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63401-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620346213, enqueue_used:2, dequeue_used:31, submit_used:1, return_used:19745, callback_enqueue_used:4, callback_dequeue_used:24, callback_finish_used:-1, end_used:1742176620366160}}, part_id:-1}) [2025-03-17 09:57:00.366175] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=38][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.366482] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=38][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.366519] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=36][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:00.366529] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=10][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d1c315a, src_buf_size=1611) [2025-03-17 09:57:00.366542] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=12][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6600, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63401-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620366219, enqueue_used:1, dequeue_used:15, submit_used:2, return_used:231, callback_enqueue_used:2, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.366590] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=39][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6600, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63401-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620366219, enqueue_used:1, dequeue_used:15, submit_used:2, return_used:231, callback_enqueue_used:2, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176620366583}}, part_id:-1}) [2025-03-17 09:57:00.366722] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222511695, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3c1c20}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:00.366760] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:23918}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634348} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:00.366784] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=20] end stmt(ret=0, tx_id=9634348, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3c1c20, tx_id:{txid:9634348}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222511695, assoc_session_id:3222511695, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176620345701, active_ts:1742176620345701, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176650345694, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464460351357, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620342535}], exec_info_reap_ts:1742176620342534, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620342531, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222511695, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3c1c20}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.366925] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=0] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634348}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176650345694, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620345701}{self_ls_id:{id:1}, session_id:3222511695, associated_session_id:3222511695, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634348}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464460351357, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620345701, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634348} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.366982] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=52] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634348}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176620366912, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:00.366998] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63401-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3c1c20, tx_id:{txid:9634348}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222511695, assoc_session_id:3222511695, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176620345701, active_ts:1742176620345701, commit_ts:-1, finish_ts:1742176620366448, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176650345694, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464460351357, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620342535}], exec_info_reap_ts:1742176620342534, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620342531, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.368167] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF912-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.368185] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF912-0-0] [lt=16][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.368193] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF912-0-0] [lt=5][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36d, hex: D3C9971A80300600, int: 1742176620366291}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c375, hex: D3C9971A80300600, int: 1742176620366291}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.368339] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF912-0-0] [lt=44][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.368669] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF912-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3338}, to_seq_no={branch:0, seq:2280}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397094} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.370085] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117877][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF913-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1742176620369629000, start_id=1742176620369629000, end_id=1742176620369629001) [2025-03-17 09:57:00.370576] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859703919}], scn_array=[cnt:1, {val:1742176620332897000, v:0}], curr_size=588, accum_size=8156, time_cost=37452) [2025-03-17 09:57:00.370635] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703811}], scn_array=[cnt:1, {val:1742176620157204002, v:0}], curr_size=113, accum_size=1211, time_cost=10100) [2025-03-17 09:57:00.376005] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D22B6C55-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.376057] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B0940-00063020D22B6C55-0-0] [lt=32][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.376536] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B0940-00063020D22B6C55-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8455a0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222503893, assoc_session_id:3222503893, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620374943, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620373879, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ade0bb118, id:3222503893, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace8455a0}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:00.381072] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.383433] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463517741}], scn_array=[cnt:1, {val:1742176620346762001, v:0}], curr_size=587, accum_size=5144, time_cost=17418) [2025-03-17 09:57:00.387235] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=31][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:0, dropped:20210, tid:3539497}]) [2025-03-17 09:57:00.393110] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF914-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.393168] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF914-0-0] [lt=56][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.393180] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF914-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36d, hex: 702D981A80300600, int: 1742176620391792}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c375, hex: 702D981A80300600, int: 1742176620391792}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.393304] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF914-0-0] [lt=69][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.393824] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF914-0-0] [lt=40] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1242}, to_seq_no={branch:0, seq:178}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397095} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.405942] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=42] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40294632652, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:00.414268] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463518879}], scn_array=[cnt:1, {val:1742176620395191000, v:0}], curr_size=551, accum_size=6246, time_cost=18537) [2025-03-17 09:57:00.415655] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.415760] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=30] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.415881] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859704507}], scn_array=[cnt:1, {val:1742176620395191000, v:0}], curr_size=587, accum_size=8743, time_cost=19964) [2025-03-17 09:57:00.416018] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.416073] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=20] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.416115] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.416130] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.416559] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF916-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.416594] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF916-0-0] [lt=33][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.416608] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF916-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36e, hex: A288981A80300600, int: 1742176620415138}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c376, hex: A288981A80300600, int: 1742176620415138}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.416748] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF916-0-0] [lt=71][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.416865] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=27] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.417135] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF916-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1387}, to_seq_no={branch:0, seq:325}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397096} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.432752] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=24] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:00.446525] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=27] new mysql sessid created(conn.sessid_=3222520192, support_ssl=false) [2025-03-17 09:57:00.446569] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=23] sm conn init succ(conn.sessid_=3222520192, sess.client_addr_="127.0.0.1:9400") [2025-03-17 09:57:00.446582] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=9] accept one succ(*s={this:0x7f7af9aa4030, session_id:3222520192, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:303}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.446643] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=21] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.446676] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117620][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=13] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.446765] INFO load_privilege_info (obmp_connect.cpp:582) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F0568499-0-0] [lt=45] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.447306] INFO [SERVER] process (obmp_connect.cpp:523) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F0568499-0-0] [lt=22] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222520192, proxy_sessid=0, client_sessid=3222520192, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.447699] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118870][T1_L0_G10000][T1][YB420A0B093E-00062FB8C01ACB01-0-0] [lt=6] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12240, add_timestamp:1742176620447686}) [2025-03-17 09:57:00.447941] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A4-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12240, locations=[{table_id:12240, partition_id:12240, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620447930, sql_renew_time:1742176620447930, is_mark_fail:false}]) [2025-03-17 09:57:00.447963] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A4-0-0] [lt=22] success to process renew task(task={tenant_id:1, table_id:12240, add_timestamp:1742176620447686}, locations=[{table_id:12240, partition_id:12240, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620447930, sql_renew_time:1742176620447930, is_mark_fail:false}]) [2025-03-17 09:57:00.448598] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=0] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:00.456982] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=29] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=7) [2025-03-17 09:57:00.458385] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3222520105, support_ssl=false) [2025-03-17 09:57:00.458527] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=118] sm conn init succ(conn.sessid_=3222520105, sess.client_addr_="127.0.0.1:9402") [2025-03-17 09:57:00.458549] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=17] accept one succ(*s={this:0x7f79906eadb0, session_id:3222520105, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:285}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:00.458600] INFO [SERVER] extract_user_tenant (obmp_connect.cpp:83) [117619][sql_nio0][T0][Y0-0000000000000000-0-0] [lt=24] username and tenantname(user_name=ocp_monitor, tenant_name=) [2025-03-17 09:57:00.458629] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117619][sql_nio0][T1][Y0-0000000000000000-0-0] [lt=11] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:00.458693] INFO load_privilege_info (obmp_connect.cpp:582) [117811][T1_MysqlQueueTh][T1][Y0-00062FB7F07684F3-0-0] [lt=31] no tenant name set, use default tenant name(tenant_name=sys) [2025-03-17 09:57:00.459456] INFO [SERVER] process (obmp_connect.cpp:523) [117811][T1_MysqlQueueTh][T1][Y0-00062FB7F07684F3-0-0] [lt=14] MySQL LOGIN(direct_client_ip="127.0.0.1", client_ip=127.0.0.1, tenant_name=sys, tenant_id=1, user_name=ocp_monitor, host_name=%, sessid=3222520105, proxy_sessid=0, client_sessid=3222520105, from_proxy=false, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=762509, proxy_capability=0, use_ssl=false, c/s protocol="OB_MYSQL_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:00.459835] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755301-0-0] [lt=0] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12099, add_timestamp:1742176620459819}) [2025-03-17 09:57:00.459877] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118869][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755301-0-0] [lt=23] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12101, add_timestamp:1742176620459861}) [2025-03-17 09:57:00.459886] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A5-0-0] [lt=11] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12099, locations=[{table_id:12099, partition_id:12099, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620459872, sql_renew_time:1742176620459872, is_mark_fail:false}]) [2025-03-17 09:57:00.459916] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A5-0-0] [lt=29] success to process renew task(task={tenant_id:1, table_id:12099, add_timestamp:1742176620459819}, locations=[{table_id:12099, partition_id:12099, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620459872, sql_renew_time:1742176620459872, is_mark_fail:false}]) [2025-03-17 09:57:00.459949] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A6-0-0] [lt=22] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12101, locations=[{table_id:12101, partition_id:12101, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620459944, sql_renew_time:1742176620459944, is_mark_fail:false}]) [2025-03-17 09:57:00.459965] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A6-0-0] [lt=17] success to process renew task(task={tenant_id:1, table_id:12101, add_timestamp:1742176620459861}, locations=[{table_id:12101, partition_id:12101, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620459944, sql_renew_time:1742176620459944, is_mark_fail:false}]) [2025-03-17 09:57:00.463982] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463519430}], scn_array=[cnt:1, {val:1742176620418667000, v:0}], curr_size=551, accum_size=6797, time_cost=44752) [2025-03-17 09:57:00.464079] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=31] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7968, ref_cnt_:1, raw_buf_:0x7f7aa4274150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63402-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176620449863, enqueue_used:2, dequeue_used:35, submit_used:12, return_used:14152, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:00.464130] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=50][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.464163] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=32][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.464183] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:00.464191] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7968, ref_cnt_:1, raw_buf_:0x7f7aa4274150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63402-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176620449863, enqueue_used:2, dequeue_used:35, submit_used:12, return_used:14152, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.464222] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=19][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7968, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63402-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176620449863, enqueue_used:2, dequeue_used:35, submit_used:12, return_used:14152, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176620464212}}, part_id:-1}) [2025-03-17 09:57:00.465853] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=31] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}]) [2025-03-17 09:57:00.466461] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF918-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.466493] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF918-0-0] [lt=29][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.466505] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF918-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36e, hex: B14C991A80300600, int: 1742176620465329}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c376, hex: B14C991A80300600, int: 1742176620465329}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.466621] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF918-0-0] [lt=63][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.467000] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF918-0-0] [lt=55] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2130}, to_seq_no={branch:0, seq:1066}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397097} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.476009] INFO pn_print_diag_info (group.c:755) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO client:0x7f7b1edf34a8_"192.168.1.64:2882"_"192.168.1.64:2882"_284_1741767244416421_1, write_queue=0/0, write=1037668/1122390673, read=1037668/72183456486, doing=0, done=1037668, write_time=1062417752, read_time=27277205, process_time=27358446 [2025-03-17 09:57:00.476038] INFO pn_print_diag_info (group.c:755) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=27] PNIO client:0x7f7af01ea048_"192.168.1.65:2882"_"192.168.1.65:2882"_225_1741767243805215_1, write_queue=0/0, write=887224/1050425642, read=887224/71086379530, doing=0, done=887224, write_time=906676320, read_time=26744635, process_time=26817960 [2025-03-17 09:57:00.476046] INFO pn_print_diag_info (group.c:755) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO client:0x7f7af01c74a8_"192.168.1.62:2882"_"192.168.1.62:2882"_212_1741316880935969_1, write_queue=0/0, write=2566047/2998982137, read=2566046/636548200, doing=0, done=2566047, write_time=2621924602, read_time=200793, process_time=385436 [2025-03-17 09:57:00.476052] INFO pn_print_diag_info (group.c:755) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x7f7b1ec05258_"192.168.1.63:2882"_"192.168.1.63:2882"_183_1741316878932599_1, write_queue=0/0, write=2196905/2406724645, read=2196904/162613496461, doing=1, done=2196904, write_time=2249313318, read_time=55311405, process_time=55491809 [2025-03-17 09:57:00.476057] INFO pn_print_diag_info (group.c:768) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7af01fca98_"192.168.1.64:57140"_296_1741767245258507, write_queue=0/0, write=2748569/1125558543, read=2748569/3659951789, doing=0, done=2748569, write_time=2802782706, read_time=2041571, process_time=2197579 [2025-03-17 09:57:00.476063] INFO pn_print_diag_info (group.c:768) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7b1edf4048_"192.168.1.65:25786"_204_1741767243874164, write_queue=0/0, write=2689248/1093335614, read=2689248/3472399962, doing=0, done=2689248, write_time=2741974684, read_time=2115400, process_time=2268500 [2025-03-17 09:57:00.476097] INFO pn_print_diag_info (group.c:768) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=33] PNIO server:0x7f7af01eaa98_"192.168.1.62:16228"_213_1741316880936272, write_queue=0/0, write=2566046/636548200, read=2566047/2998982137, doing=0, done=2566047, write_time=2607872913, read_time=320937, process_time=467735 [2025-03-17 09:57:00.476103] INFO pn_print_diag_info (group.c:768) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7b1edf54a8_"192.168.1.63:41034"_202_1741316880364962, write_queue=0/0, write=5551313/3218165420, read=5551313/7143127429, doing=0, done=5551313, write_time=5656951773, read_time=4224561, process_time=4537506 [2025-03-17 09:57:00.476107] INFO pn_print_diag_info (group.c:775) [117488][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client_send:7935190/9481207996, client_queue_time=166344045, cnt=4, server_send:18653705/8072277929, server_queue_time=343147278, cnt=4 [2025-03-17 09:57:00.480751] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=21][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.480795] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=43][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:00.480803] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f797409715a, src_buf_size=1611) [2025-03-17 09:57:00.480812] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c70f8, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63402-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176620464265, enqueue_used:2, dequeue_used:18, submit_used:0, return_used:16422, callback_enqueue_used:4, callback_dequeue_used:36, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.480856] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=36][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c70f8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63402-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176620464265, enqueue_used:2, dequeue_used:18, submit_used:0, return_used:16422, callback_enqueue_used:4, callback_dequeue_used:36, callback_finish_used:-1, end_used:1742176620480851}}, part_id:-1}) [2025-03-17 09:57:00.480939] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416069}], scn_array=[cnt:1, {val:1742176620157204003, v:0}], curr_size=122, accum_size=235, time_cost=21181) [2025-03-17 09:57:00.481032] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222516413, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf263b40}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:00.481084] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:31238}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634352} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:00.481136] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=25] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:57:00.481156] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872703924}], scn_array=[cnt:1, {val:1742176620157204003, v:0}], curr_size=122, accum_size=1333, time_cost=21423) [2025-03-17 09:57:00.481114] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=26] end stmt(ret=0, tx_id=9634352, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf263b40, tx_id:{txid:9634352}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516413, assoc_session_id:3222516413, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176620448704, active_ts:1742176620448704, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176650448699, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464564031403, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620448707}], exec_info_reap_ts:1742176620448706, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620448703, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222516413, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf263b40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.481182] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=45] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:57:00.481185] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1333) [2025-03-17 09:57:00.481194] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.481217] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1333) [2025-03-17 09:57:00.481266] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:00.481273] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634352}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176650448699, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620448704}{self_ls_id:{id:1}, session_id:3222516413, associated_session_id:3222516413, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634352}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464564031403, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620449768, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634352} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.481337] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=52] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634352}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176650448699, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620448704}{self_ls_id:{id:1}, session_id:3222516413, associated_session_id:3222516413, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634352}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464564031403, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620449768, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634352} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.481377] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=35] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634352}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176620481270, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:00.481393] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf263b40, tx_id:{txid:9634352}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516413, assoc_session_id:3222516413, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176620448704, active_ts:1742176620448704, commit_ts:-1, finish_ts:1742176620481013, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176650448699, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464564031403, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620448707}], exec_info_reap_ts:1742176620448706, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620448703, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.481477] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63402-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:00.481548] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=24] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:00.481574] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=21] temporary file module use no blocks [2025-03-17 09:57:00.481591] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=16] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:00.481595] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=18] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:00.481605] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:00.481619] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=21] temporary file module use no blocks [2025-03-17 09:57:00.481631] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=11] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:00.481619] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863115373332, flush_io_finished_ret_:0, flush_io_finished_round_:859378, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:00.481643] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:00.481661] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=21] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:00.481657] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863115373385, flush_io_finished_ret_:0, flush_io_finished_round_:580428, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:00.481682] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=17] temporary file module use no blocks [2025-03-17 09:57:00.481691] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=8] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:00.481703] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=11] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:00.481719] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=15] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863115373451, flush_io_finished_ret_:0, flush_io_finished_round_:580429, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:00.483887] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463519981}], scn_array=[cnt:1, {val:1742176620468180000, v:0}], curr_size=553, accum_size=7350, time_cost=15107) [2025-03-17 09:57:00.486360] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91A-0-0] [lt=8][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.486383] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91A-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.486392] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36e, hex: 4398991A80300600, int: 1742176620484675}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c376, hex: 4398991A80300600, int: 1742176620484675}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.486506] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91A-0-0] [lt=64][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.486799] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117877][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF91A-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2753}, to_seq_no={branch:0, seq:2138}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397098} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.511817] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=37] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:57:00.511867] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=69] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:57:00.521141] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463520534}], scn_array=[cnt:1, {val:1742176620488544000, v:0}], curr_size=553, accum_size=7903, time_cost=32502) [2025-03-17 09:57:00.521209] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859705094}], scn_array=[cnt:1, {val:1742176620395191001, v:0}], curr_size=122, accum_size=8865, time_cost=10896) [2025-03-17 09:57:00.521245] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8865) [2025-03-17 09:57:00.521260] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=18, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:57:00.521288] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=55] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=9, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:57:00.521312] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=27745, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:57:00.521319] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=17, avg_log_gen_to_freeze_time=3264, avg_log_gen_to_submit_time=3277, avg_log_submit_to_flush_time=24446, avg_log_submit_to_slide_time=24467) [2025-03-17 09:57:00.521335] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=17, stat_interval=1000000, avg (count/cost)=44, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:57:00.521344] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=28] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8865, hit_cnt=17, read_cnt=17, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:00.521932] INFO pn_print_diag_info (group.c:755) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x7f7ae97ea048_"192.168.1.64:2882"_"192.168.1.64:2882"_282_1741767244409041_1, write_queue=0/0, write=1010417/1103300245, read=1010417/70296332022, doing=0, done=1010417, write_time=1034650189, read_time=26360863, process_time=26445562 [2025-03-17 09:57:00.521955] INFO pn_print_diag_info (group.c:755) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=21] PNIO client:0x7f7ae9760048_"192.168.1.65:2882"_"192.168.1.65:2882"_209_1741767243222731_1, write_queue=0/0, write=861190/1015414567, read=861190/69211459571, doing=0, done=861190, write_time=880904275, read_time=25696496, process_time=25768931 [2025-03-17 09:57:00.521964] INFO pn_print_diag_info (group.c:755) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7ae97eca58_"192.168.1.62:2882"_"192.168.1.62:2882"_234_1741316886941360_1, write_queue=0/0, write=2496886/2917510064, read=2496886/619094826, doing=0, done=2496886, write_time=2552080428, read_time=196619, process_time=385306 [2025-03-17 09:57:00.521972] INFO pn_print_diag_info (group.c:755) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=7] PNIO client:0x7f7b1ecfaa98_"192.168.1.63:2882"_"192.168.1.63:2882"_191_1741316878943638_1, write_queue=1/592, write=2138649/2357732170, read=2138649/158303787914, doing=1, done=2138649, write_time=2189208501, read_time=53348407, process_time=53521939 [2025-03-17 09:57:00.521982] INFO pn_print_diag_info (group.c:768) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO server:0x7f7ae97fe048_"192.168.1.64:20994"_310_1741767246064843, write_queue=0/0, write=2674129/1121553007, read=2674129/3557571888, doing=0, done=2674129, write_time=2727800239, read_time=2020755, process_time=2173775 [2025-03-17 09:57:00.521992] INFO pn_print_diag_info (group.c:768) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO server:0x7f7ae97ef468_"192.168.1.65:25840"_269_1741767244188757, write_queue=0/0, write=2615804/1089692794, read=2615804/3383133269, doing=0, done=2615804, write_time=2667603735, read_time=2067431, process_time=2217371 [2025-03-17 09:57:00.522001] INFO pn_print_diag_info (group.c:768) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO server:0x7f7ae97ed4a8_"192.168.1.62:16270"_236_1741316886941703, write_queue=0/0, write=2496886/619094826, read=2496886/2917510064, doing=0, done=2496886, write_time=2537493985, read_time=311805, process_time=454578 [2025-03-17 09:57:00.522011] INFO pn_print_diag_info (group.c:768) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=9] PNIO server:0x7f7b1edf2048_"192.168.1.63:41018"_197_1741316879612559, write_queue=0/0, write=5398147/3136399702, read=5398147/6945950736, doing=0, done=5398147, write_time=5501133864, read_time=4270950, process_time=4576343 [2025-03-17 09:57:00.522019] INFO pn_print_diag_info (group.c:775) [117493][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client_send:7720769/9248952037, client_queue_time=162929997, cnt=4, server_send:18146501/7910603935, server_queue_time=334887907, cnt=4 [2025-03-17 09:57:00.523842] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add16b90 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397099} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1069} write_epoch=1, prev=this=0x7f78069b9628 trans_version={val:1742176254372575000, v:0} scn={val:1742176254372575000, v:0} tx_id={txid:14396689} prev=0x7f778f2ca800 next=(nil) modify_count=63 acc_checksum=1993215173 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3424} write_epoch=3, snapshot_version={val:1742176620522818000, v:0}, *this={this=0x7f7778ffb5c8 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x7f78069b9628 latest_compact_node=0x7f79adc31990 max_trans_version={val:1742176559595495000, v:0} max_trans_id=14397031 max_elr_trans_version={val:1742176559595495000, v:0} max_elr_trans_id=14397031 latest_compact_ts=1742175889738355 last_compact_cnt=59 total_trans_node_cnt=64 max_modify_scn={val:1742176254372575000, v:0} min_modify_scn={val:1742078732499628000, v:0}}) [2025-03-17 09:57:00.523889] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=46][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add16b90 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397099} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1069} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650423047, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650423047, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620522818000, v:0}, tx_id:{txid:14397099}, scn:{branch:0, seq:1066}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397099}, tx_desc:{this:0x7f7ae6a517e0, tx_id:{txid:14397099}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176620522818, commit_ts:-1, finish_ts:-1, timeout_us:29999988, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620521993, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1069}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.523969] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=79][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397099}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1069}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:00.523982] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=13][errcode=0] primary key duplication conflict(key={"BIGINT":1005},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant file data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397099} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:00.524013] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=30][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.524022] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36e, hex: 0A2A9A1A80300600, int: 1742176620521994}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c376, hex: 0A2A9A1A80300600, int: 1742176620521994}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37e, hex: 1D86110500000000, int: 85034525}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c386, hex: 00D0C00600000000, int: 113299456}]}) [2025-03-17 09:57:00.524105] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=58][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.524412] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0074-0-0] [lt=0][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.524483] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0074-0-0] [lt=69][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.524532] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0074-0-0] [lt=31][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.524557] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF91C-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2141}, to_seq_no={branch:0, seq:1067}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397099} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.525031] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0074-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace869c60, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222520216, assoc_session_id:3222520216, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620522818, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620522817, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222520216, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace869c60}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.532866] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1B-0-0] [lt=3][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.532911] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1B-0-0] [lt=13][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.533293] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1B-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6b4e6c0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222520514, assoc_session_id:3222520514, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620531357, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620531356, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7b00e56038, id:3222520514, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6b4e6c0}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:00.536339] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=1][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add16e08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397100} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2089} write_epoch=1, prev=this=0x7f786a0b4fb0 trans_version={val:1742149862622327000, v:0} scn={val:1742149862622327000, v:0} tx_id={txid:13986947} prev=0x7f786a0b4e00 next=(nil) modify_count=9 acc_checksum=2020689202 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:5303} write_epoch=3, snapshot_version={val:1742176620535633000, v:0}, *this={this=0x7f7778ffb860 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x7f786a0b4fb0 latest_compact_node=0x7f77e96cc8e8 max_trans_version={val:1742176559640263000, v:0} max_trans_id=14397032 max_elr_trans_version={val:1742176559640263000, v:0} max_elr_trans_id=14397032 latest_compact_ts=1742148278688686 last_compact_cnt=5 total_trans_node_cnt=10 max_modify_scn={val:1742149862622327000, v:0} min_modify_scn={val:1742148033102563000, v:0}}) [2025-03-17 09:57:00.536387] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add16e08 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397100} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2089} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650435551, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650435551, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620535633000, v:0}, tx_id:{txid:14397100}, scn:{branch:0, seq:2086}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397100}, tx_desc:{this:0x7f7ae6ae9400, tx_id:{txid:14397100}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176620535633, commit_ts:-1, finish_ts:-1, timeout_us:29999988, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620533727, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:2089}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.536483] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=95][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397100}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:2089}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:00.536500] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant file data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397100} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:00.536530] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=29][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.536543] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c690, local_datums:0x7f782b80c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E742066696C652064617461, cstr: tenant file data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c36e, hex: DB5B9A1A80300600, int: 1742176620534747}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c376, hex: DB5B9A1A80300600, int: 1742176620534747}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c37e, hex: 8DCA010100000000, int: 16894605}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c386, hex: 00105B0100000000, int: 22745088}]}) [2025-03-17 09:57:00.536634] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=61][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.536927] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF91E-0-0] [lt=36] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2970}, to_seq_no={branch:0, seq:2087}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397100} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.556132] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=10] can close safely, do destroy(*s={this:0x7f7af9b46db0, session_id:3222520260, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:301}, err:5, last_decode_time:1742176620142256, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:692, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:57:00.556202] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=52] unlock session of tenant(conn.sessid_=3222520260, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:57:00.556216] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=13] connection close(sessid=3222520260, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB7F06F5716-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:14, pkt_rec[0]:{send:obp_mysql_header_{len_:11, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{receive:obp_mysql_header_{len_:441, rec_:441, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[2]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[4]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_mysql_header_{pkt_num_:2, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{receive:obp_mysql_header_{len_:137, rec_:137, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[8]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_mysql_header_{pkt_num_:4, seq_:6}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{send:obp_mysql_header_{len_:9, seq_:7}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[11]:{send:obp_mysql_header_{pkt_num_:10, seq_:17}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:18}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:57:00.556252] INFO [SERVER] run (obmp_disconnect.cpp:83) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=35] free session successfully(sessid=3222520260) [2025-03-17 09:57:00.556276] INFO [SERVER] run (obmp_disconnect.cpp:90) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=23] mark session id unused(sessid=3222520260) [2025-03-17 09:57:00.556816] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=6][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=488, pos=315) [2025-03-17 09:57:00.557476] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=29][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=12, len=168, pos=151) [2025-03-17 09:57:00.557960] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=20][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=43, len=285, pos=256) [2025-03-17 09:57:00.558683] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=44, pos=15) [2025-03-17 09:57:00.559098] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=16][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=40, len=271, pos=244) [2025-03-17 09:57:00.559593] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=194, pos=168) [2025-03-17 09:57:00.560440] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=15][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=35, len=231, pos=202) [2025-03-17 09:57:00.561440] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=21][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=75, pos=48) [2025-03-17 09:57:00.562573] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5718-0-0] [lt=17][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=10, len=143, pos=123) [2025-03-17 09:57:00.563481] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=33] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=124784640) [2025-03-17 09:57:00.563525] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=32] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:00.564072] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=58][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463521640}], scn_array=[cnt:1, {val:1742176620537768000, v:0}], curr_size=553, accum_size=9009, time_cost=25382) [2025-03-17 09:57:00.566335] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF920-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.566367] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF920-0-0] [lt=29][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.566377] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF920-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77a2e0c6a0, local_datums:0x7f77a2e0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77a2e0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f77a2e0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c370, hex: 74CF9A1A80300600, int: 1742176620564340}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c378, hex: 74CF9A1A80300600, int: 1742176620564340}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77a2e0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.566503] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF920-0-0] [lt=77][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.566833] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF920-0-0] [lt=9] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3190}, to_seq_no={branch:0, seq:2126}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397101} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.581163] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.585533] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463522193}], scn_array=[cnt:1, {val:1742176620546497000, v:0}], curr_size=587, accum_size=9596, time_cost=21347) [2025-03-17 09:57:00.601312] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:00.606321] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=43] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40286244044, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:00.614769] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118838][T1_L1_G10000][T1][YB420A0B0941-00063020D4A68952-0-0] [lt=7][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.615586] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:999) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=11] can close safely, do destroy(*s={this:0x7f7ad287c030, session_id:3222489765, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:true, reader:{fd:300}, err:5, last_decode_time:1742176620101648, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:2616, pending_flag:1, may_handling_flag:false, handler_close_flag:false}) [2025-03-17 09:57:00.615666] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=38] unlock session of tenant(conn.sessid_=3222489765, proxy_sessid=0, conn.tenant_id_=1) [2025-03-17 09:57:00.615710] INFO [SERVER] run (obmp_disconnect.cpp:83) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=22] free session successfully(sessid=3222489765) [2025-03-17 09:57:00.615725] INFO [SERVER] run (obmp_disconnect.cpp:90) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=14] mark session id unused(sessid=3222489765) [2025-03-17 09:57:00.615691] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117621][sql_nio2][T0][Y0-0000000000000000-0-0] [lt=24] connection close(sessid=3222489765, proxy_sessid=0, tenant_id=1, from_proxy=false, from_java_client=false, c/s protocol="OB_MYSQL_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB7F0DFECA4-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:0, cur_pkt_pos_:44, pkt_rec[12]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{receive:obp_mysql_header_{len_:146, rec_:146, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[14]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[17]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[18]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[19]:{receive:obp_mysql_header_{len_:166, rec_:166, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[20]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[21]:{send:obp_mysql_header_{pkt_num_:1, seq_:3}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[22]:{send:obp_mysql_header_{len_:9, seq_:4}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[23]:{send:obp_mysql_header_{pkt_num_:1, seq_:5}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[24]:{send:obp_mysql_header_{len_:9, seq_:6}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[25]:{receive:obp_mysql_header_{len_:507, rec_:507, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[26]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[27]:{send:obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[28]:{send:obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[29]:{send:obp_mysql_header_{pkt_num_:2, seq_:12}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[30]:{send:obp_mysql_header_{len_:9, seq_:13}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[31]:{receive:obp_mysql_header_{len_:383, rec_:383, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[32]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[33]:{send:obp_mysql_header_{pkt_num_:530, seq_:11}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[34]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[35]:{send:obp_mysql_header_{pkt_num_:6, seq_:14}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[36]:{send:obp_mysql_header_{len_:9, seq_:15}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[37]:{receive:obp_mysql_header_{len_:633, rec_:633, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[38]:{send:obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[39]:{send:obp_mysql_header_{pkt_num_:132, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[40]:{send:obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[41]:{send:obp_mysql_header_{pkt_num_:3, seq_:11}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[42]:{send:obp_mysql_header_{len_:9, seq_:12}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[43]:{receive:obp_mysql_header_{len_:1, rec_:1, seq_:0}, pkt_name:"COM_QUIT", obp_mysql_header_.type_:1, obp_mysql_header_.is_send_:0}], disconnect_state=3) [2025-03-17 09:57:00.616506] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118882][T1_L0_G10000][T1][YB420A0B093F-00062FB99FB767B3-0-0] [lt=28][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.617233] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118856][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755302-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12223, add_timestamp:1742176620617222}) [2025-03-17 09:57:00.617248] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A7-0-0] [lt=15] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12223, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.617399] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A7-0-0] [lt=18] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12223, locations=[{table_id:12223, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.617465] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A7-0-0] [lt=65] success to process renew task(task={tenant_id:1, table_id:12223, add_timestamp:1742176620617222}, locations=[{table_id:12223, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}, {table_id:12223, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620617380, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.619176] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118838][T1_L1_G10000][T1][YB420A0B0940-00063020D05ECB2A-0-0] [lt=23][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.619386] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118856][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755302-0-0] [lt=25][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.619564] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [118875][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEACA0-0-0] [lt=17] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12325, add_timestamp:1742176620619549}) [2025-03-17 09:57:00.619590] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A8-0-0] [lt=44] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12325, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:00.619815] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A8-0-0] [lt=18] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12325, locations=[{table_id:12325, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.619855] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A8-0-0] [lt=39] success to process renew task(task={tenant_id:1, table_id:12325, add_timestamp:1742176620619549}, locations=[{table_id:12325, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}, {table_id:12325, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176620619803, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:00.621459] INFO [SERVER] inner_get_next_row (ob_all_virtual_px_target_monitor.cpp:70) [118885][T1_L0_G10000][T1][YB420A0B093E-00062FB95E755302-0-0] [lt=25] get_px_target finish(ret=-4008) [2025-03-17 09:57:00.626118] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=8] run rewrite rule refresh task(rule_mgr_->tenant_id_=1) [2025-03-17 09:57:00.626859] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118857][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEACA0-0-0] [lt=22][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.627417] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=39] get number(ret=0, service_type_=0, range=1, base_id=1742176620626755000, start_id=1742176620626755000, end_id=1742176620626755001) [2025-03-17 09:57:00.629523] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=29] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=31196160) [2025-03-17 09:57:00.629599] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=29] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=78, size_used=0, mem_used=0) [2025-03-17 09:57:00.629666] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=14] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:00.630212] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463522780}], scn_array=[cnt:1, {val:1742176620567906000, v:0}], curr_size=557, accum_size=10153, time_cost=44565) [2025-03-17 09:57:00.630326] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=45][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.630356] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=28][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.630379] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=22][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:00.630417] INFO [LIB] stat (utility.h:1210) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=24] [APPLY STAT CB APPEND COST TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=29, this=0x7f7ad1022280, extra_info="") [2025-03-17 09:57:00.630388] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7e30, ref_cnt_:1, raw_buf_:0x7f79740b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63403-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176620602502, enqueue_used:2, dequeue_used:127, submit_used:1, return_used:27666, callback_enqueue_used:4, callback_dequeue_used:21, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.630470] INFO [LIB] stat (utility.h:1210) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=53] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=22900, this=0x7f7ad10224b0, extra_info="") [2025-03-17 09:57:00.630476] INFO [LIB] stat (utility.h:1210) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=0, this=0x7f7ad10226e0, extra_info="") [2025-03-17 09:57:00.630481] INFO [LIB] stat (utility.h:1210) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=4] [APPLY STAT CB EXECUTE TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=-488, this=0x7f7ad1022910, extra_info="") [2025-03-17 09:57:00.630481] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=81][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7e30, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63403-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176620602502, enqueue_used:2, dequeue_used:127, submit_used:1, return_used:27666, callback_enqueue_used:4, callback_dequeue_used:21, callback_finish_used:-1, end_used:1742176620630473}}, part_id:-1}) [2025-03-17 09:57:00.630500] INFO [LIB] stat (utility.h:1210) [117837][T1_ApplySrv1][T1][Y0-0000000000000000-0-0] [lt=19] [APPLY STAT CB TOTAL TIME](cur_stat_count=47, stat_interval=5000000, avg (count/cost)=22442, this=0x7f7ad1022b40, extra_info="") [2025-03-17 09:57:00.630835] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=22][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:00.630959] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222518960, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3c7d40}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:00.631000] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=28] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:29082}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634355} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:00.631025] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=22] end stmt(ret=0, tx_id=9634355, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3c7d40, tx_id:{txid:9634355}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518960, assoc_session_id:3222518960, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176620601926, active_ts:1742176620601926, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176650601921, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464716651032, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620600863}], exec_info_reap_ts:1742176620600862, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620600859, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222518960, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3c7d40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.631166] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=1][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634355}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176650601921, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620601926}{self_ls_id:{id:1}, session_id:3222518960, associated_session_id:3222518960, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634355}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464716651032, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620601926, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634355} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.631229] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=55] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634355}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176650601921, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620601926}{self_ls_id:{id:1}, session_id:3222518960, associated_session_id:3222518960, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634355}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464716651032, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620601926, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634355} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.631267] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=35] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634355}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176620631163, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:00.631281] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=12] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3c7d40, tx_id:{txid:9634355}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222518960, assoc_session_id:3222518960, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176620601926, active_ts:1742176620601926, commit_ts:-1, finish_ts:1742176620631010, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176650601921, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464716651032, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176620600863}], exec_info_reap_ts:1742176620600862, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620600859, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.631356] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63403-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:00.632843] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT WRITE LOG COST TIME](cur_stat_count=18, stat_interval=1000000, avg (count/cost)=22710, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:57:00.632865] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=21][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859705804}], scn_array=[cnt:1, {val:1742176620595526000, v:0}], curr_size=588, accum_size=1176, time_cost=36474) [2025-03-17 09:57:00.632992] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF922-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.633014] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF922-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.633026] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF922-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c6a0, local_datums:0x7f782b80c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c370, hex: D0D69B1A80300600, int: 1742176620631760}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c378, hex: D0D69B1A80300600, int: 1742176620631760}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.632922] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=9919) [2025-03-17 09:57:00.633166] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF922-0-0] [lt=51][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.633296] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=52] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1742176620395191000, v:0}, new_max_version={val:1742176620395191000, v:0}, last_min_version={val:1742176620195447000, v:0}, last_max_version={val:1742176620195447000, v:0}, total_time_us=38453, delta_us=238104) [2025-03-17 09:57:00.633323] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1742176620395191000, v:0}, min_version={val:1742176620395191000, v:0}, max_version={val:1742176620395191000, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:57:00.633522] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF922-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1383}, to_seq_no={branch:0, seq:323}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397102} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.635093] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.635183] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=26] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.635228] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117878][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF923-0-0] [lt=41] get number(ret=0, service_type_=0, range=1, base_id=1742176620634169000, start_id=1742176620634169000, end_id=1742176620634169001) [2025-03-17 09:57:00.635288] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636011] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636077] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=40] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636075] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636170] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636179] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=32] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636184] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.636255] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=14] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:00.637350] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118838][T1_L1_G10000][T1][YB420A0B093E-00062FB7F0AEACA0-0-0] [lt=34][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.637704] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118856][T1_L0_G10000][T1][YB420A0B093F-00062FB99FB767B3-0-0] [lt=28][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.640409] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118871][T1_L0_G10000][T1][YB420A0B0941-00063020D4A68952-0-0] [lt=8][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.641566] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=32] gc stale ls task succ [2025-03-17 09:57:00.644291] WDIAG [SERVER] inner_get_next_row (ob_information_query_response_time.cpp:107) [118838][T1_L1_G10000][T1][YB420A0B0940-00063020D05ECB2A-0-0] [lt=23][errcode=-4008] fail to execute(ret=-4008) [2025-03-17 09:57:00.657105] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=6) [2025-03-17 09:57:00.658178] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=11] gts statistics(tenant_id=1003, gts_rpc_cnt=10, get_gts_cache_cnt=601, get_gts_with_stc_cnt=62, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=62, try_wait_gts_elapse_cnt=0) [2025-03-17 09:57:00.660095] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=63] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2025-03-17 09:57:00.660130] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1211) [2025-03-17 09:57:00.660170] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1211, avg_group_log_size=121) [2025-03-17 09:57:00.660179] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1211) [2025-03-17 09:57:00.662178] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [118875][T1_L0_G10000][T1][YB420A0B093E-00062FB7F0AEACA0-0-0] [lt=24][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:00.662553] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463523883}], scn_array=[cnt:1, {val:1742176620634169000, v:0}], curr_size=557, accum_size=11256, time_cost=27205) [2025-03-17 09:57:00.664790] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF924-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.664824] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF924-0-0] [lt=32][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.664834] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF924-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f782b80c6a0, local_datums:0x7f782b80c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f782b80c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f782b80c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c370, hex: 90509C1A80300600, int: 1742176620662928}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c378, hex: 90509C1A80300600, int: 1742176620662928}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f782b80c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.664925] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF924-0-0] [lt=43][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.665281] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF924-0-0] [lt=33] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2133}, to_seq_no={branch:0, seq:1841}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397103} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.666470] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571A-0-0] [lt=20][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=189, pos=169) [2025-03-17 09:57:00.667211] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571A-0-0] [lt=24][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=440, pos=300) [2025-03-17 09:57:00.667909] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571A-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=9, len=117, pos=91) [2025-03-17 09:57:00.674130] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=41] gc stale ls task succ [2025-03-17 09:57:00.680936] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=54][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416313}], scn_array=[cnt:1, {val:1742176620657896000, v:0}], curr_size=122, accum_size=479, time_cost=20738) [2025-03-17 09:57:00.681225] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.682592] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872704168}], scn_array=[cnt:1, {val:1742176620657896000, v:0}], curr_size=122, accum_size=244, time_cost=22474) [2025-03-17 09:57:00.682655] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463524440}], scn_array=[cnt:1, {val:1742176620666462000, v:0}], curr_size=557, accum_size=11813, time_cost=15351) [2025-03-17 09:57:00.685230] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF926-0-0] [lt=8][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.685263] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF926-0-0] [lt=30][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.685275] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF926-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c6a0, local_datums:0x7f781dc0c6a0}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 18, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206261636B75702064617461, cstr: tenant backup data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c370, hex: CC9F9C1A80300600, int: 1742176620683212}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c378, hex: CC9F9C1A80300600, int: 1742176620683212}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c380, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c388, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.685392] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF926-0-0] [lt=56][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.685948] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF926-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3488}, to_seq_no={branch:0, seq:2154}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397104} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.688527] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=30][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:438, tid:3539397}]) [2025-03-17 09:57:00.703819] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:817) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=33] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}) [2025-03-17 09:57:00.704216] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463524997}], scn_array=[cnt:1, {val:1742176620686689000, v:0}], curr_size=557, accum_size=12370, time_cost=16504) [2025-03-17 09:57:00.711735] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:00.712684] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=36] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.213025097846984863e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=606, Throttle Trigger(MB)=363, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.308525433948665295e+02) [2025-03-17 09:57:00.713580] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=0][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.713626] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=44][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.713650] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=23][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:00.713660] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9c90, ref_cnt_:1, raw_buf_:0x7f7a22814150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63404-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713004, enqueue_used:1, dequeue_used:19, submit_used:2, return_used:531, callback_enqueue_used:3, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.713714] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=44][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9c90, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63404-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713004, enqueue_used:1, dequeue_used:19, submit_used:2, return_used:531, callback_enqueue_used:3, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176620713707}}, part_id:-1}) [2025-03-17 09:57:00.713720] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=29][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713004, enqueue_used:1, dequeue_used:19, submit_used:2, return_used:531, callback_enqueue_used:3, callback_dequeue_used:16, callback_finish_used:173, end_used:-42}}) [2025-03-17 09:57:00.713767] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=46][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:00.713780] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:00.714086] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=35][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.714111] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=25][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:00.714118] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f797409715a, src_buf_size=1611) [2025-03-17 09:57:00.714126] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8f10, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63404-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713791, enqueue_used:2, dequeue_used:16, submit_used:1, return_used:264, callback_enqueue_used:2, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:00.714186] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=54][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8f10, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63404-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713791, enqueue_used:2, dequeue_used:16, submit_used:1, return_used:264, callback_enqueue_used:2, callback_dequeue_used:8, callback_finish_used:-1, end_used:1742176620714182}}, part_id:-1}) [2025-03-17 09:57:00.714221] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176620713791, enqueue_used:2, dequeue_used:16, submit_used:1, return_used:264, callback_enqueue_used:2, callback_dequeue_used:8, callback_finish_used:131, end_used:-33}}) [2025-03-17 09:57:00.714244] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=22][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:00.714252] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=7][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:00.714278] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=25][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:00.714299] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=19][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:00.714314] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=15][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:00.714340] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=26][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:00.714403] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=62][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:57:00.714411] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=7][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:00.714418] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=6][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:00.714457] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=35][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:00.714464] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176650612872, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:00.714553] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=88][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176650612872, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f7a0e5c498c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 5A0F9D1A80300600, int: 1742176620711770}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}}) [2025-03-17 09:57:00.714630] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=74][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f7a0e5c498c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 5A0F9D1A80300600, int: 1742176620711770}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:57:00.714664] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=32][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:00.714690] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=24][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:57:00.714703] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=12][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}, ret=-4016) [2025-03-17 09:57:00.714761] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=12][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7a0e5c4ed0, local_datums:0x7f7a0e5c4b20}, datums:[col_id=0:{len: 27, flag: 0, null: 0, ptr: 0x7f7a0e5c498c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49a7, hex: 5A0F9D1A80300600, int: 1742176620711770}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49af, hex: 5A0F9D1A80300600, int: 1742176620711770}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49b7, hex: 5A0F9D1A80300600, int: 1742176620711770}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49bf, hex: 0900000000000000, int: 9}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49c7, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7a0e5c49cf, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7a0e5c49d5, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 49, flag: 0, null: 0, ptr: 0x7f7a0e5c49e6, hex: 44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029, cstr: DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7a0e5c4a17, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c4a20, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c4a28, hex: 53B0C11DF92F0600, int: 1741596852858963}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7a0e5c4a30, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7a0e5c4a41, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176650612872, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650612872, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620657896000, v:0}, tx_id:{txid:9634356}, scn:{branch:0, seq:2125}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634356}, tx_desc:{this:0x7f7acf3ee460, tx_id:{txid:9634356}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517061, assoc_session_id:3222517061, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176620712392, active_ts:1742176620712392, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176650712386, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464827132205, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176620712393}], exec_info_reap_ts:1742176620712392, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620710268, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:2126}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.714883] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=121][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:57:00.714895] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f7a0e5c498c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49a7, hex: 5A0F9D1A80300600, int: 1742176620711770}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 27, flag: 0, null: 0, ptr: 0x7f7a0e5c498c, hex: 4153594E435F4741544845525F53544154535F4A4F425F50524F43, cstr: ASYNC_GATHER_STATS_JOB_PROC}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a0e5c49a7, hex: 5A0F9D1A80300600, int: 1742176620711770}], store_rowkey:{"VARCHAR":"ASYNC_GATHER_STATS_JOB_PROC", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:00.711770"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176650612872, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:57:00.714988] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=93][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:57:00.714995] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=7][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:57:00.715008] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=6][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:00.715018] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:57:00.715048] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=25][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:00.715054] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=6][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:57:00.715063] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:00.715073] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:57:00.715079] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=6][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176650712872, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388705, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf3ee460, tx_id:{txid:9634356}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517061, assoc_session_id:3222517061, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176620712392, active_ts:1742176620712392, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176650712386, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464827132205, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2126}, last_touch_ts:1742176620712393}], exec_info_reap_ts:1742176620712392, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620710268, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634356}, scn:{branch:0, seq:2125}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176650612872, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176650612872, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176650612872, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:570, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:57:00.715980] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=901][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:57:00.715991] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:00.716003] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=11][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:00.716009] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=5][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:57:00.716015] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=5][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:57:00.716022] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=5][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:57:00.716034] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=12][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:57:00.716044] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:00.716053] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:57:00.716061] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:57:00.716071] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:00.716080] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:00.716089] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}) [2025-03-17 09:57:00.716101] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=11][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:57:00.716112] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=10][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:00.716137] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=18] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222517061, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3ee460}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:00.716195] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=37] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4260}, to_seq_no={branch:0, seq:2125}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634356} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:00.716226] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=25] end stmt(ret=0, tx_id=9634356, plain_select=false, stmt_type=2, savepoint={branch:0, seq:2125}, tx_desc={this:0x7f7acf3ee460, tx_id:{txid:9634356}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517061, assoc_session_id:3222517061, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176620712392, active_ts:1742176620712392, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176650712386, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464827132205, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2125}, last_touch_ts:1742176620712393}], exec_info_reap_ts:1742176620712392, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620710268, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222517061, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3ee460}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:00.716273] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=46][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:00.716283] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:00.716291] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=8][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:00.716306] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:00.716317] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=10][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')"}, aret=-4016, ret=-4016) [2025-03-17 09:57:00.716326] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=7][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC')) [2025-03-17 09:57:00.716347] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=10][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:57:00.716357] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC'), is_user_sql=false) [2025-03-17 09:57:00.716368] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=10][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176620711770), usec_to_time(1742176620711770), 9, usec_to_time(1742176620711770), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4153594E435F4741544845525F53544154535F4A4F425F50524F432836303030303030303029', 'SCHEDULED', usec_to_time(1741596852858963), 'ASYNC_GATHER_STATS_JOB_PROC'), conn=0x7f7aed35a3a0) [2025-03-17 09:57:00.716378] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:57:00.716470] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634356}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176650712386, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620712392}{self_ls_id:{id:1}, session_id:3222517061, associated_session_id:3222517061, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634356}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464827132205, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620712392, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634356} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.716558] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=72] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634356}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176650712386, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176620712392}{self_ls_id:{id:1}, session_id:3222517061, associated_session_id:3222517061, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634356}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409464827132205, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176620712392, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634356} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:00.716618] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=53] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634356}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176620716465, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:00.716638] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=17] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3ee460, tx_id:{txid:9634356}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517061, assoc_session_id:3222517061, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176620712392, active_ts:1742176620712392, commit_ts:-1, finish_ts:1742176620715601, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176650712386, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409464827132205, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2125}, last_touch_ts:1742176620712393}], exec_info_reap_ts:1742176620712392, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620710268, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.716732] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=34][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:57:00.716742] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:57:00.716752] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63404-0-0] [lt=9][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:00.730602] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=45][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463525554}], scn_array=[cnt:1, {val:1742176620705973000, v:0}], curr_size=590, accum_size=12960, time_cost=23584) [2025-03-17 09:57:00.730762] INFO pn_print_diag_info (group.c:755) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=8] PNIO client:0x7f7af01faa58_"192.168.1.65:2882"_"192.168.1.65:2882"_312_1741767249227780_1, write_queue=0/0, write=883850/1042292224, read=883850/70977717677, doing=0, done=883850, write_time=903486704, read_time=26241510, process_time=26315462 [2025-03-17 09:57:00.730784] INFO pn_print_diag_info (group.c:755) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=20] PNIO client:0x7f7af01fc048_"192.168.1.64:2882"_"192.168.1.64:2882"_287_1741767244428691_1, write_queue=0/0, write=1038737/1129004168, read=1038737/72172292704, doing=0, done=1038737, write_time=1063036082, read_time=27015607, process_time=27099249 [2025-03-17 09:57:00.730791] INFO pn_print_diag_info (group.c:755) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=6] PNIO client:0x7f7af01fd4a8_"192.168.1.62:2882"_"192.168.1.62:2882"_239_1741316890935158_1, write_queue=0/0, write=2567057/3015188026, read=2567057/636552125, doing=0, done=2567057, write_time=2623783988, read_time=202500, process_time=389761 [2025-03-17 09:57:00.730797] INFO pn_print_diag_info (group.c:755) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO client:0x7f7b1ec6ea98_"192.168.1.63:2882"_"192.168.1.63:2882"_188_1741316878934349_1, write_queue=1/350, write=2199173/2415400003, read=2199173/162742276218, doing=1, done=2199173, write_time=2251462885, read_time=54852013, process_time=55036874 [2025-03-17 09:57:00.730802] INFO pn_print_diag_info (group.c:768) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7b1edfea98_"192.168.1.64:20964"_302_1741767246049920, write_queue=0/0, write=2749324/1130664253, read=2749324/3660996015, doing=0, done=2749324, write_time=2803503729, read_time=2061394, process_time=2217923 [2025-03-17 09:57:00.730808] INFO pn_print_diag_info (group.c:768) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO server:0x7f7af01c6048_"192.168.1.65:25802"_238_1741767244184933, write_queue=0/0, write=2688154/1092975245, read=2688154/3464831965, doing=0, done=2688154, write_time=2741672481, read_time=2050759, process_time=2204992 [2025-03-17 09:57:00.730813] INFO pn_print_diag_info (group.c:768) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7af01fe048_"192.168.1.62:31780"_240_1741316890935375, write_queue=0/0, write=2567057/636552125, read=2567057/3015188026, doing=0, done=2567057, write_time=2609227008, read_time=320135, process_time=467102 [2025-03-17 09:57:00.730818] INFO pn_print_diag_info (group.c:768) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO server:0x7f7af01fa048_"192.168.1.63:41046"_223_1741316882615034, write_queue=0/0, write=5548550/3255795688, read=5548550/7140389994, doing=0, done=5548550, write_time=5654840364, read_time=4285040, process_time=4599562 [2025-03-17 09:57:00.730822] INFO pn_print_diag_info (group.c:775) [117490][pnio1][T0][Y0-0000000000000000-0-0] [lt=4] PNIO client_send:7935341/9490614995, client_queue_time=167102131, cnt=4, server_send:18652954/8082124201, server_queue_time=344622008, cnt=4 [2025-03-17 09:57:00.732721] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539259][T1004_ArcSrv][T1003][YB420A0B093E-00062FB7F8654125-0-0] [lt=55] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7ad2288200, tx_id:{txid:9634357}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516157, assoc_session_id:3222516157, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1742176620731649, active_ts:1742176620731649, commit_ts:-1, finish_ts:1742176620732672, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176650731643, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620731648, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.733002] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92A-0-0] [lt=17][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.733028] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92A-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.733045] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92A-0-0] [lt=13][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 655B9D1A80300600, int: 1742176620731237}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 655B9D1A80300600, int: 1742176620731237}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.733166] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92A-0-0] [lt=61][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.733521] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF92A-0-0] [lt=43] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2519}, to_seq_no={branch:0, seq:2169}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397106} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.733825] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539259][T1004_ArcSrv][T1003][YB420A0B093E-00062FB7F8654125-0-0] [lt=57] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7ad2263b40, tx_id:{txid:9634359}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222512027, assoc_session_id:3222512027, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:3, alloc_ts:1742176620732672, active_ts:1742176620732672, commit_ts:-1, finish_ts:1742176620733749, timeout_us:29999996, lock_timeout_us:-1, expire_ts:1742176650732668, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620732677, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:00.746323] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63405-0-0] [lt=9] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:57:00.767954] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=23][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859706392}], scn_array=[cnt:1, {val:1742176620732672000, v:0}], curr_size=587, accum_size=1763, time_cost=34837) [2025-03-17 09:57:00.768172] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463526144}], scn_array=[cnt:1, {val:1742176620734807000, v:0}], curr_size=553, accum_size=13513, time_cost=32564) [2025-03-17 09:57:00.770075] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add17ea8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397107} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1074} write_epoch=1, prev=this=0x7f79add06060 trans_version={val:1742176559797065000, v:0} scn={val:1742176559797065000, v:0} tx_id={txid:14397039} prev=0x7f778f2cd7a8 next=(nil) modify_count=3589 acc_checksum=952243171 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3483} write_epoch=3, snapshot_version={val:1742176620768837000, v:0}, *this={this=0x7f7778ffc910 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add06060 latest_compact_node=0x7f79adcf3fc0 max_trans_version={val:1742176559797065000, v:0} max_trans_id=14397039 max_elr_trans_version={val:1742176559797065000, v:0} max_elr_trans_id=14397039 latest_compact_ts=1742176498944920 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176559797065000, v:0} min_modify_scn={val:1742067221995549000, v:0}}) [2025-03-17 09:57:00.770115] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add17ea8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397107} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1074} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650669368, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650669368, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620768837000, v:0}, tx_id:{txid:14397107}, scn:{branch:0, seq:1071}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397107}, tx_desc:{this:0x7f7ae6a77f00, tx_id:{txid:14397107}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176620769904, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620768383, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1074}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.770188] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=72][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397107}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1074}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:00.770199] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=10][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant clog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397107} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:00.770221] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.770232] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 7EEC9D1A80300600, int: 1742176620768382}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 7EEC9D1A80300600, int: 1742176620768382}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: F7CAC64200000000, int: 1120324343}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: F7CAC64200000000, int: 1120324343}]}) [2025-03-17 09:57:00.770294] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.770578] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF92C-0-0] [lt=23] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1522}, to_seq_no={branch:0, seq:1072}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397107} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.772343] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117877][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF92D-0-0] [lt=33] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=14936) [2025-03-17 09:57:00.778252] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=31] run rewrite rule refresh task(rule_mgr_->tenant_id_=1004) [2025-03-17 09:57:00.778650] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=33] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:00.779587] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=21] get number(ret=0, service_type_=0, range=1, base_id=1742176620779531000, start_id=1742176620779531000, end_id=1742176620779531001) [2025-03-17 09:57:00.780988] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=59][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872704290}], scn_array=[cnt:1, {val:1742176620657896001, v:0}], curr_size=122, accum_size=366, time_cost=20835) [2025-03-17 09:57:00.781006] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416435}], scn_array=[cnt:1, {val:1742176620657896001, v:0}], curr_size=122, accum_size=601, time_cost=20828) [2025-03-17 09:57:00.781286] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.783235] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:685) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=28] clear dtl interm result cost(us)(clear_cost=1660, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2025-03-17 09:57:00.783588] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463526697}], scn_array=[cnt:1, {val:1742176620746477000, v:0}], curr_size=586, accum_size=14099, time_cost=15347) [2025-03-17 09:57:00.806647] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=7] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40292535500, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:00.812453] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=13] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:57:00.812492] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=32] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:57:00.812507] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=13] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=8484) [2025-03-17 09:57:00.817490] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463527283}], scn_array=[cnt:1, {val:1742176620772041000, v:0}], curr_size=846, accum_size=14945, time_cost=33828) [2025-03-17 09:57:00.819408] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add182e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397108} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:552} write_epoch=1, prev=this=0x7f79add064a0 trans_version={val:1742176559821579000, v:0} scn={val:1742176559821579000, v:0} tx_id={txid:14397040} prev=0x7f79add062f0 next=(nil) modify_count=3589 acc_checksum=3209582010 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4613} write_epoch=3, snapshot_version={val:1742176620817856000, v:0}, *this={this=0x7f7778ffcd70 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add064a0 latest_compact_node=0x7f79adcf44c8 max_trans_version={val:1742176559821579000, v:0} max_trans_id=14397040 max_elr_trans_version={val:1742176559821579000, v:0} max_elr_trans_id=14397040 latest_compact_ts=1742176498979306 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176559821579000, v:0} min_modify_scn={val:1742067222049768000, v:0}}) [2025-03-17 09:57:00.819487] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=78][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add182e8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397108} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:552} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650718748, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650718748, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620817856000, v:0}, tx_id:{txid:14397108}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397108}, tx_desc:{this:0x7f7ae6a57900, tx_id:{txid:14397108}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176620818920, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620818369, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:552}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.819565] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=76][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397108}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:552}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:00.819575] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=11][errcode=0] primary key duplication conflict(key={"BIGINT":1005},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant clog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397108} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:00.819596] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.819607] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: C2AF9E1A80300600, int: 1742176620818370}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: C2AF9E1A80300600, int: 1742176620818370}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: A8D8EAF200000000, int: 4075477160}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: A8D8EAF200000000, int: 4075477160}]}) [2025-03-17 09:57:00.819686] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=53][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.820036] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117888][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF92E-0-0] [lt=8] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1074}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397108} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.829330] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859706979}], scn_array=[cnt:1, {val:1742176620795500000, v:0}], curr_size=588, accum_size=2351, time_cost=32815) [2025-03-17 09:57:00.857711] WDIAG [STORAGE.TRANS] inner_lock_for_read (ob_tx_data_functor.cpp:342) [117410][TimerWK3][T1][YB420A0B093E-00062FB95067DE5A-0-0] [lt=27][errcode=-6004] lock_for_read need retry(ret=-6004, tx_data={tx_id:{txid:14855937}, ref_cnt:1, state:"RUNNING", commit_version:{val:1742176620849902000, v:0}, start_scn:{val:1742176620849902000, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}, lock_for_read_arg_={mvcc_acc_ctx_:{type:1, abs_lock_timeout_ts:1742176650757570, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620849902000, v:0}, tx_id:{txid:0}, scn:0}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:0}, tx_desc:NULL, tx_ctx:null, mem_ctx:null, tx_scn:0, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, data_trans_id_:{txid:14855937}, data_sql_sequence_:{branch:0, seq:3}, read_latest_:false, read_uncommitted_:false, scn_:{val:1742176620849902000, v:0}}, tx_cc_ctx={state:10, prepare_version:{val:1742176620849902000, v:0}}) [2025-03-17 09:57:00.860323] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=18] QSched: |cfg: root:1,x,0 net_in:1,1249M,0 net_out:1,1248M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:368K/84:15275 disk:368K/84:15275 t1m2:160K/36:14695 t1m2g0:160K/36:14695 t1004m2:92K/22:15480 t1004m2g0:92K/22:15480 t1003m2:116K/26:15907 t1003m2g0:116K/26:15907 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:57:00.866483] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=35] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=5) [2025-03-17 09:57:00.881354] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.881871] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463528129}], scn_array=[cnt:1, {val:1742176620822127000, v:0}], curr_size=847, accum_size=15792, time_cost=59526) [2025-03-17 09:57:00.883890] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=35][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add18560 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2142} write_epoch=1, prev=this=0x7f79add068e0 trans_version={val:1742176559846168000, v:0} scn={val:1742176559846168000, v:0} tx_id={txid:14397041} prev=0x7f79add06730 next=(nil) modify_count=3589 acc_checksum=3690471051 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3717} write_epoch=3, snapshot_version={val:1742176620882967000, v:0}, *this={this=0x7f7778ffd1d0 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add068e0 latest_compact_node=0x7f79adcf49d0 max_trans_version={val:1742176559846168000, v:0} max_trans_id=14397041 max_elr_trans_version={val:1742176559846168000, v:0} max_elr_trans_id=14397041 latest_compact_ts=1742176499000657 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176559846168000, v:0} min_modify_scn={val:1742067222084962000, v:0}}) [2025-03-17 09:57:00.883950] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=59][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add18560 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397109} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2142} write_epoch=1, ctx={this:0x7f7af4cd3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650783145, tablet_id:{id:309}, table_iter:0x7f7af4cd1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650783145, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176620882967000, v:0}, tx_id:{txid:14397109}, scn:{branch:0, seq:1597}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397109}, tx_desc:{this:0x7f7ace875ea0, tx_id:{txid:14397109}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176620882967, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176620881371, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:2142}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:00.884045] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=94][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397109}, data:0x7f7af4cd0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:2142}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:00.884062] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=16][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant clog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397109} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:00.884096] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=32][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.884111] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=12][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781780c690, local_datums:0x7f781780c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781780c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781780c35e, hex: 74656E616E7420636C6F672064617461, cstr: tenant clog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c36e, hex: 12AA9F1A80300600, int: 1742176620882450}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c376, hex: 12AA9F1A80300600, int: 1742176620882450}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c37e, hex: 5F93076000000000, int: 1611109215}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781780c386, hex: 5F93076000000000, int: 1611109215}]}) [2025-03-17 09:57:00.884226] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117898][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=70][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.884571] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF930-0-0] [lt=33] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2643}, to_seq_no={branch:0, seq:1599}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397109} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.887455] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416557}], scn_array=[cnt:1, {val:1742176620779531000, v:0}], curr_size=122, accum_size=723, time_cost=27249) [2025-03-17 09:57:00.887522] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=8, avg_log_submit_to_flush_time=18521, avg_log_submit_to_slide_time=18535) [2025-03-17 09:57:00.887562] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1333, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:00.893977] WDIAG [LIB.TIME] init (ob_timezone_info.cpp:39808) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=74][errcode=-4005] init twice(ret=-4005) [2025-03-17 09:57:00.894013] WDIAG [SHARE] init (ob_time_zone_info_manager.cpp:97) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=35][errcode=-4005] init tz info map failed(ret=-4005, tenant_id_=1004) [2025-03-17 09:57:00.894022] WDIAG [SHARE] fetch_time_zone_info (ob_time_zone_info_manager.cpp:171) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-4005] init failed(ret=-4005) [2025-03-17 09:57:00.894029] WDIAG [SERVER.OMT] operator() (ob_tenant_timezone_mgr.cpp:56) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4005] fail to update time zone info(ret=-4005) [2025-03-17 09:57:00.896854] WDIAG [LIB.TIME] init (ob_timezone_info.cpp:39808) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=5][errcode=-4005] init twice(ret=-4005) [2025-03-17 09:57:00.896874] WDIAG [SHARE] init (ob_time_zone_info_manager.cpp:97) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4005] init tz info map failed(ret=-4005, tenant_id_=1006) [2025-03-17 09:57:00.896882] WDIAG [SHARE] fetch_time_zone_info (ob_time_zone_info_manager.cpp:171) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-4005] init failed(ret=-4005) [2025-03-17 09:57:00.896888] WDIAG [SERVER.OMT] operator() (ob_tenant_timezone_mgr.cpp:56) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=6][errcode=-4005] fail to update time zone info(ret=-4005) [2025-03-17 09:57:00.930346] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463529522}], scn_array=[cnt:1, {val:1742176620887153000, v:0}], curr_size=847, accum_size=17185, time_cost=42658) [2025-03-17 09:57:00.930471] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859707567}], scn_array=[cnt:1, {val:1742176620795500001, v:0}], curr_size=122, accum_size=2473, time_cost=19820) [2025-03-17 09:57:00.932670] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF932-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.932698] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF932-0-0] [lt=25][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.932721] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF932-0-0] [lt=19][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36f, hex: 4E69A01A80300600, int: 1742176620931406}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c377, hex: 4E69A01A80300600, int: 1742176620931406}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37f, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c387, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.932861] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF932-0-0] [lt=67][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.933205] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF932-0-0] [lt=43] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2588}, to_seq_no={branch:0, seq:1525}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397110} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.933312] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=16, avg_log_batch_cnt=1, total_group_log_size=8247, avg_group_log_size=515) [2025-03-17 09:57:00.934000] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=35][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:57:00.934020] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:57:00.939369] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=15] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:0, threads_idle_time:1312804415218, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:57:00.939473] INFO [LIB] do_thread_recycle (threads.cpp:163) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=42] recycle one thread(this=0x7f7b319af170, total=4, remain=3) [2025-03-17 09:57:00.939497] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=24] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:0, threads_idle_time:1312804415218, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:57:00.940520] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=11] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1003) [2025-03-17 09:57:00.940549] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=27] start lib cache evict(tenant_id=1003, mem_hold=75497472, mem_limit=85899345, cache_obj_num=228, cache_node_num=228) [2025-03-17 09:57:00.940558] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1003, cache_evict_num=0, mem_hold=75497472, mem_limit=85899345, cache_obj_num=228, cache_node_num=228) [2025-03-17 09:57:00.940565] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:00.941205] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF934-0-0] [lt=13][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.941234] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF934-0-0] [lt=26][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.941246] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF934-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 848AA01A80300600, int: 1742176620939908}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 848AA01A80300600, int: 1742176620939908}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 00B00C0000000000, int: 831488}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 00B00C0000000000, int: 831488}]}) [2025-03-17 09:57:00.941370] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF934-0-0] [lt=64][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.941783] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117878][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF934-0-0] [lt=26] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2580}, to_seq_no={branch:0, seq:1540}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397111} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.943315] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=8] Dumping All Cache Objs(alloc_obj_list.count()=228, alloc_obj_list=[{obj_id:33320, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184745}, {obj_id:35162, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191402}, {obj_id:35168, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202676}, {obj_id:35176, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202872}, {obj_id:35185, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194566}, {obj_id:35193, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190237}, {obj_id:35195, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193489}, {obj_id:35251, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179204}, {obj_id:35253, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:35255, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177976}, {obj_id:37271, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201056}, {obj_id:37386, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233847}, {obj_id:38041, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237416}, {obj_id:38401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194846}, {obj_id:38952, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:38998, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39000, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39002, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:39007, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193123}, {obj_id:39008, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185047}, {obj_id:39340, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182356}, {obj_id:39341, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181004}, {obj_id:39342, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179416}, {obj_id:39348, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196117}, {obj_id:39351, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201780}, {obj_id:39367, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235254}, {obj_id:39369, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179880}, {obj_id:39402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199220}, {obj_id:39540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192539}, {obj_id:39697, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226000}, {obj_id:39909, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182976}, {obj_id:39919, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181840}, {obj_id:39920, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180292}, {obj_id:39921, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187396}, {obj_id:39928, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:39930, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:39932, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:39937, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:39938, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:39939, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:39942, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:39945, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:39946, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:39947, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183646}, {obj_id:39949, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182873}, {obj_id:39950, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184278}, {obj_id:40084, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40106, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206365}, {obj_id:40107, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223790}, {obj_id:40108, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180350}, {obj_id:40109, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263305}, {obj_id:40110, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331790}, {obj_id:40111, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40129, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183490}, {obj_id:40158, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180900}, {obj_id:40159, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188646}, {obj_id:40160, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204918}, {obj_id:40193, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192613}, {obj_id:40275, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200576}, {obj_id:40276, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40277, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180004}, {obj_id:40280, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195104}, {obj_id:40281, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40282, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:263647}, {obj_id:40284, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:40285, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182846}, {obj_id:40286, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181614}, {obj_id:40287, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199088}, {obj_id:40288, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198336}, {obj_id:40289, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216980}, {obj_id:40290, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203406}, {obj_id:40291, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40292, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192722}, {obj_id:40293, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204622}, {obj_id:40294, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181130}, {obj_id:40295, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194163}, {obj_id:40296, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194023}, {obj_id:40297, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181347}, {obj_id:40298, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178480}, {obj_id:40299, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179672}, {obj_id:40300, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177992}, {obj_id:40301, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184955}, {obj_id:40302, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199077}, {obj_id:40303, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180472}, {obj_id:40304, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207773}, {obj_id:40305, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40306, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191932}, {obj_id:40307, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185119}, {obj_id:40308, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40309, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40310, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:40311, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203354}, {obj_id:40312, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178272}, {obj_id:40313, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40314, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182249}, {obj_id:40315, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248104}, {obj_id:40316, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40317, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:40318, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188408}, {obj_id:40324, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:40325, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:40327, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191558}, {obj_id:40330, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186614}, {obj_id:40331, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186470}, {obj_id:40332, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182421}, {obj_id:40334, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194093}, {obj_id:40338, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183137}, {obj_id:40339, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179672}, {obj_id:40340, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180218}, {obj_id:40341, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207773}, {obj_id:40342, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:40343, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177800}, {obj_id:40344, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182821}, {obj_id:40345, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189868}, {obj_id:40346, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180883}, {obj_id:40347, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200944}, {obj_id:40348, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178491}, {obj_id:40349, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179802}, {obj_id:40350, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183073}, {obj_id:40351, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:40353, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:40354, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:40356, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181614}, {obj_id:40357, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:40358, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:40359, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:40360, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178320}, {obj_id:40361, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179563}, {obj_id:40362, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178320}, {obj_id:40363, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:40364, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:40368, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193848}, {obj_id:40401, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192816}, {obj_id:40402, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208968}, {obj_id:40403, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40405, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230424}, {obj_id:40406, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300745}, {obj_id:40407, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:40409, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:40410, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297044}, {obj_id:40411, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40470, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276547}, {obj_id:40471, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251949}, {obj_id:40472, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219417}, {obj_id:40473, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280167}, {obj_id:40474, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224140}, {obj_id:40475, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221591}, {obj_id:40476, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:40477, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212536}, {obj_id:40478, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:40479, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189930}, {obj_id:40480, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183988}, {obj_id:40481, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204926}, {obj_id:40482, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235402}, {obj_id:40483, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182233}, {obj_id:40484, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179094}, {obj_id:40485, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199080}, {obj_id:40486, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179054}, {obj_id:40487, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179088}, {obj_id:40488, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203106}, {obj_id:40489, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226815}, {obj_id:40490, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215316}, {obj_id:40491, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206428}, {obj_id:40492, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40493, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205754}, {obj_id:40494, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206492}, {obj_id:40495, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264163}, {obj_id:40496, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194128}, {obj_id:40497, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40498, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221082}, {obj_id:40499, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190664}, {obj_id:40500, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191156}, {obj_id:40501, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:40502, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:40503, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:40504, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:40505, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180726}, {obj_id:40506, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181131}, {obj_id:40507, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203744}, {obj_id:40508, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:295360}, {obj_id:40509, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203218}, {obj_id:40510, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:441835}, {obj_id:40511, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184744}, {obj_id:40512, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221820}, {obj_id:40513, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381057}, {obj_id:40514, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:376803}, {obj_id:40515, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274484}, {obj_id:40516, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250503}, {obj_id:40517, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200576}, {obj_id:40518, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185905}, {obj_id:40519, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:269349}, {obj_id:40520, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:282801}, {obj_id:40521, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:253249}, {obj_id:40522, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268180}, {obj_id:40523, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193179}, {obj_id:40524, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200691}, {obj_id:40525, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180248}, {obj_id:40526, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:40527, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182846}, {obj_id:40528, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:40530, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:40531, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:40532, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180844}, {obj_id:40533, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:40534, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181059}, {obj_id:40535, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:40536, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:40537, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:40538, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:40539, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:40540, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:40541, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:40542, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:40543, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268481}, {obj_id:40544, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246352}, {obj_id:40545, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178818}, {obj_id:40546, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264469}, {obj_id:40547, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243566}, {obj_id:40548, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202676}, {obj_id:40549, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194233}, {obj_id:40550, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180200}, {obj_id:40551, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192722}, {obj_id:40552, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272495}, {obj_id:40553, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249127}, {obj_id:40554, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192277}, {obj_id:40555, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203356}, {obj_id:40556, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191402}, {obj_id:40557, tenant_id:1003, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184214}]) [2025-03-17 09:57:00.943864] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=548] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:00.947920] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT APPEND COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:57:00.959802] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=20] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:929, free_obj_hold_cnt:159, allocator used:4186624, allocator total:4439040}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:149, free_obj_hold_cnt:931, allocator used:70838280, allocator total:70838280}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():0, full allocator used:0, full allocator total:0}, tablets_mem=75277320, tablets_mem_limit=1288490180, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:33, free_obj_hold_cnt:50, allocator used:169984, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=1078) [2025-03-17 09:57:00.960262] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:57:00.960300] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:57:00.960317] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:57:00.971341] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [117761][T1_Occam][T1][YB420A0B093E-00062FB860254125-0-0] [lt=27] ====== tenant freeze timer task ====== [2025-03-17 09:57:00.981442] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:00.982171] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=53] [PALF STAT WRITE LOG COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=24672, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:57:00.982198] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=26][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463530922}], scn_array=[cnt:1, {val:1742176620942489000, v:0}], curr_size=551, accum_size=18289, time_cost=38681) [2025-03-17 09:57:00.982215] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=18289) [2025-03-17 09:57:00.982228] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=22, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:57:00.982251] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=18289) [2025-03-17 09:57:00.982297] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=31, stat_interval=1000000, avg (count/cost)=0, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:57:00.982292] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=1][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.982313] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=21][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:00.982334] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:00.982335] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=14] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=8, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:57:00.982347] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=29891, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:57:00.982342] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8b68, ref_cnt_:1, raw_buf_:0x7f7ac862e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB727D5F461-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620972961, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:9291, callback_enqueue_used:2, callback_dequeue_used:15, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:00.982379] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=30, stat_interval=1000000, avg (count/cost)=45, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:57:00.982378] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=27][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8b68, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB727D5F461-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620972961, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:9291, callback_enqueue_used:2, callback_dequeue_used:15, callback_finish_used:-1, end_used:1742176620982371}}, part_id:-1}) [2025-03-17 09:57:00.982402] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=0][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620972961, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:9291, callback_enqueue_used:2, callback_dequeue_used:15, callback_finish_used:113, end_used:-32}}) [2025-03-17 09:57:00.982475] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=70][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:00.982489] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:00.983055] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16930, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:57:00.983073] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872704956}], scn_array=[cnt:1, {val:1742176620779531001, v:0}], curr_size=122, accum_size=1032, time_cost=22717) [2025-03-17 09:57:00.983093] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:57:00.983120] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:57:00.983137] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=17168, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:57:00.983165] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=31, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:57:00.984331] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF936-0-0] [lt=9][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:00.984354] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF936-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:00.984367] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF936-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36f, hex: B831A11A80300600, int: 1742176620982712}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c377, hex: B831A11A80300600, int: 1742176620982712}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37f, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c387, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:00.984516] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF936-0-0] [lt=73][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:00.984843] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF936-0-0] [lt=40] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1280}, to_seq_no={branch:0, seq:823}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397112} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:00.986388] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [117741][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=29] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2025-03-17 09:57:00.988947] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=24][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:00.988984] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=36][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:00.988956] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416679}], scn_array=[cnt:1, {val:1742176620779531001, v:0}], curr_size=122, accum_size=845, time_cost=28595) [2025-03-17 09:57:00.988990] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaac31510, src_buf_size=295) [2025-03-17 09:57:00.988997] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7890, ref_cnt_:1, raw_buf_:0x7f7aaac30150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB727D5F461-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620982499, enqueue_used:1, dequeue_used:15, submit_used:2, return_used:6397, callback_enqueue_used:2, callback_dequeue_used:28, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:00.989039] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=34][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7890, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093F-00062FB727D5F461-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620982499, enqueue_used:1, dequeue_used:15, submit_used:2, return_used:6397, callback_enqueue_used:2, callback_dequeue_used:28, callback_finish_used:-1, end_used:1742176620989033}}, part_id:-1}) [2025-03-17 09:57:00.989047] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac25a2080, time_log:{begin_ts:1742176620982499, enqueue_used:1, dequeue_used:15, submit_used:2, return_used:6397, callback_enqueue_used:2, callback_dequeue_used:28, callback_finish_used:124, end_used:-35}}) [2025-03-17 09:57:00.989093] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=42][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:00.989110] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=14][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac25a2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:00.989139] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=29][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac25a2080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:00.989181] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=37][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:00.989198] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=17][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:00.989227] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=28][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f78178262f0, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f7a8d8839e0, last_rowkey:0x7f7a8d883b48, discrete_rowkey_array:0x7f7a8d883c68, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883a70, hex: EB03000000000000, int: 1003}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883aa8, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b08, hex: C8F2AD7DB79BD4E7, int: -1741595943495667000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b20, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f7a8d883963, ps_node_array:0x7f7a8d8839a0, data_buf:0x7f7a8d883963, data_buf_size:1031}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:00.989297] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=70][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=2, prefetched_cnt=0) [2025-03-17 09:57:00.989308] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=10][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:00.989317] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=8][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:00.989333] WDIAG [STORAGE] inner_open (ob_sstable_multi_version_row_iterator.cpp:162) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=10][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:00.989344] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=10][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176650870232, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc5e050, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176620947793000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7a989be090, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176620972290, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:00.989786] WDIAG [STORAGE] scan (ob_sstable.cpp:427) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=441][errcode=-4016] Fail to open row scanner(ret=-4016, param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176650870232, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc5e050, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176620947793000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7a989be090, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176620972290, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}, key_range={start_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814610, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=1:MIN_OBJ], store_rowkey:{"BIGINT":1},MIN}, end_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814690, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=1:MAX_OBJ], store_rowkey:{"BIGINT":1},MAX}, group_idx:0, border_flag:{flag:0}}, *this={ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2025-03-17 09:57:00.990091] WDIAG [STORAGE] construct_iters (ob_multiple_scan_merge.cpp:198) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=302][errcode=-4016] Fail to get iterator(ret=-4016, i=1, *iter_pram={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}) [2025-03-17 09:57:00.990317] WDIAG [STORAGE] open (ob_multiple_scan_merge.cpp:60) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=225][errcode=-4016] fail to construct iters(ret=-4016) [2025-03-17 09:57:00.990322] WDIAG [STORAGE] init_and_open_scan_merge_iter_ (ob_table_scan_iterator.cpp:651) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] Fail to open multiple merge iterator(ret=-4016) [2025-03-17 09:57:00.990328] WDIAG [STORAGE] open_iter (ob_table_scan_iterator.cpp:558) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] init and open scan merge iterator failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:00.990332] WDIAG [STORAGE] init (ob_table_scan_iterator.cpp:355) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] fail to open iter(ret=-4016, cached_iter_node_=NULL, *this={}) [2025-03-17 09:57:00.990338] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2373) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] failed to init table scan iterator, (ret=-4016) [2025-03-17 09:57:00.990343] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2379) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=4][errcode=-4016] failed to do table scan(ret=-4016, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176650870232, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176620972290, need_update_tablet_param:false}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2025-03-17 09:57:00.990574] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:674) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=230][errcode=-4016] failed to do table scan(ret=-4016, &iter=0x7f7b0cc5d3d0, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176650870232, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176620972290, need_update_tablet_param:false}) [2025-03-17 09:57:00.990829] WDIAG [STORAGE] table_scan (ob_access_service.cpp:371) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=254][errcode=-4016] Fail to scan table, (ret=-4016, ls_id={id:1}, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176650870232, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176620972290, need_update_tablet_param:false}) [2025-03-17 09:57:00.991107] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_iter.cpp:83) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=277][errcode=-4016] fail to scan table(scan_param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176650870232, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176620972290, need_update_tablet_param:false}, ret=-4016) [2025-03-17 09:57:00.991388] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:569) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=280][errcode=-4016] iter tree failed to do table scan(ret=-4016) [2025-03-17 09:57:00.991393] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=5][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:00.991398] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:102) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=4][errcode=-4016] start das task failed(ret=-4016, *task_wrapper.at(i)={parent:{tenant_id:1, task_id:212095870, op_type:1, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7ace88c2c0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222521395, assoc_session_id:3222521395, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620972291, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620972290, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, snapshot:{this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, tablet_id:{id:345}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:345}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:345, related_table_ids:[cnt:0], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:1}, in_retry:0, partition_id:345, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:, plan_line_id:0}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:345, access_column_ids:[cnt:13, 18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1741278087142176, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, pd_expr_spec:{calc_exprs:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], access_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:16, pushdown_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}], is_get:false, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0, rowkey_exprs:[cnt:0], table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, doc_id_idx:-1, vec_vid_idx:-1, index_merge_idx:-1, pre_query_range:[], is_index_merge:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:-1, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1742176650870232, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tsc_monitor_info:{*io_read_bytes_:295*ssstore_read_bytes_:0*ssstore_read_row_cnt_:0*memstore_read_row_cnt_:0}, key_ranges:[], ss_key_ranges:[], mbr_filters:[], scan_op_id:18446744073709551615, scan_rows_size:-1, das_tasks_key:{fake_unique_id_:0, timestamp_:0}}, scan_range:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}}) [2025-03-17 09:57:00.991748] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=348][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:00.991757] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=8][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:00.991762] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:260) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=4][errcode=-4016] failed to execute all das task(ret=-4016) [2025-03-17 09:57:00.991766] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2699) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=3][errcode=-4016] execute all das scan task failed(ret=-4016) [2025-03-17 09:57:00.991773] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1822) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] fail to do table scan(ret=-4016) [2025-03-17 09:57:00.991780] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2588) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] failed to init before get row(ret=-4016) [2025-03-17 09:57:00.991787] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:00.991794] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=6][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f781780bfb0}, id=0, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:00.991804] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=10][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:00.991812] WDIAG [SQL.EXE] sync_send_result (ob_remote_executor_processor.cpp:286) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:00.991820] WDIAG [SQL.EXE] execute_remote_plan (ob_remote_executor_processor.cpp:485) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=7][errcode=-4016] sync send result failed(ret=-4016) [2025-03-17 09:57:00.991840] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=6] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace88c2c0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222521395, assoc_session_id:3222521395, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176620972291, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176620972290, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7af25061f8, id:3222521395, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace88c2c0}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:00.991877] WDIAG [SQL.EXE] execute_with_sql (ob_remote_executor_processor.cpp:716) [357553][T1_L0_G19][T1][YB420A0B093F-00062FB727D5F461-0-0] [lt=32][errcode=-4016] execute remote plan failed(ret=-4016, task={tenant_schema_version:1741661297536408, sys_schema_version:1741661297536408, runner_svr:"192.168.1.62:2882", ctrl_svr:"192.168.1.63:2882", task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.63:2882", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:18446744073709551615}, task_id:0, task_cnt:0}, remote_sql_info:{use_ps:true, is_batched_stmt:false, is_original_ps_mode:false, ps_param_cnt:2, remote_sql:"SELECT * FROM __all_ls_status where tenant_id = ? and flag like ?", ps_params:[{obj:{"BIGINT":1}, accuracy:{length:1, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}, {obj:{"VARCHAR":"%DUPLICATE%", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}, accuracy:{length:11, precision:1, scale:-1}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}}]}, snapshot:{this:0x7f7817804208, valid:false, source:0, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}}, exec_ctx_.get_das_ctx().get_snapshot()={this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176620947793000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}) [2025-03-17 09:57:00.996255] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:57:01.003982] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=7][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463532060}], scn_array=[cnt:1, {val:1742176620986354000, v:0}], curr_size=555, accum_size=1142, time_cost=14944) [2025-03-17 09:57:01.006017] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF938-0-0] [lt=11][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.006038] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF938-0-0] [lt=19][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.006048] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF938-0-0] [lt=7][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36f, hex: EF88A11A80300600, int: 1742176621005039}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c377, hex: EF88A11A80300600, int: 1742176621005039}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37f, hex: 0060000000000000, int: 24576}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c387, hex: 0060000000000000, int: 24576}]}) [2025-03-17 09:57:01.006138] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF938-0-0] [lt=39][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.006498] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF938-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2141}, to_seq_no={branch:0, seq:1074}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397113} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.007131] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40286244044, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:01.011973] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=18] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:57:01.012027] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:57:01.012712] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA586-0-0] [lt=23][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:01.018460] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175478][T1_L0_G5][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1742176621017339000, start_id=1742176621017339000, end_id=1742176621017339001) [2025-03-17 09:57:01.019052] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f778f2ce2f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397114} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:1070} write_epoch=1, prev=this=0x7f79adcadae8 trans_version={val:1742176254770321000, v:0} scn={val:1742176254770321000, v:0} tx_id={txid:14396704} prev=0x7f79adcad930 next=(nil) modify_count=63 acc_checksum=3964224393 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:4319} write_epoch=3, snapshot_version={val:1742176621017339000, v:0}, *this={this=0x7f7778ffe090 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x7f79adcadae8 latest_compact_node=0x7f78069ada10 max_trans_version={val:1742176559918451000, v:0} max_trans_id=14397046 max_elr_trans_version={val:1742176559918451000, v:0} max_elr_trans_id=14397046 latest_compact_ts=1742175890227505 last_compact_cnt=59 total_trans_node_cnt=64 max_modify_scn={val:1742176254770321000, v:0} min_modify_scn={val:1742078732708827000, v:0}}) [2025-03-17 09:57:01.019100] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=47][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f778f2ce2f8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397114} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:1070} write_epoch=1, ctx={this:0x7f7af49cf658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650918181, tablet_id:{id:309}, table_iter:0x7f7af49cdb48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650918181, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621017339000, v:0}, tx_id:{txid:14397114}, scn:{branch:0, seq:480}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397114}, tx_desc:{this:0x7f7acafc3c80, tx_id:{txid:14397114}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621018408, commit_ts:-1, finish_ts:-1, timeout_us:29999990, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621017339, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b95d950, mem_ctx:0x7f7b0b95eed0, tx_scn:{branch:0, seq:1070}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.019181] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=80][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397114}, data:0x7f7af49cc508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1070}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.019192] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=10][errcode=0] primary key duplication conflict(key={"BIGINT":1005},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant local data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397114} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.019213] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.019222] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f77c0e0c690, local_datums:0x7f77c0e0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f77c0e0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f77c0e0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c36f, hex: DBBAA11A80300600, int: 1742176621017819}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c377, hex: DBBAA11A80300600, int: 1742176621017819}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c37f, hex: 00707B0700000000, int: 125530112}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f77c0e0c387, hex: 00707B0700000000, int: 125530112}]}) [2025-03-17 09:57:01.019289] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117908][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.019643] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93A-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2133}, to_seq_no={branch:0, seq:481}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397114} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.024653] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=34][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add191b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397115} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:2123} write_epoch=1, prev=this=0x7f786a0b7738 trans_version={val:1742149863129766000, v:0} scn={val:1742149863129766000, v:0} tx_id={txid:13986962} prev=0x7f786a0b7668 next=(nil) modify_count=11 acc_checksum=17217599 version=1742067188266264 type=1 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:18} write_epoch=0, snapshot_version={val:1742176621023717000, v:0}, *this={this=0x7f7778ffe330 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x7f786a0b7738 latest_compact_node=0x7f786a0b7738 max_trans_version={val:1742176559958772000, v:0} max_trans_id=14397047 max_elr_trans_version={val:1742176559958772000, v:0} max_elr_trans_id=14397047 latest_compact_ts=1742149863158469 last_compact_cnt=11 total_trans_node_cnt=12 max_modify_scn={val:1742149863129766000, v:0} min_modify_scn={val:1742071728589861000, v:0}}) [2025-03-17 09:57:01.024695] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add191b8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397115} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:89} seq_no={branch:0, seq:2123} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176650924042, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176650924042, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621023717000, v:0}, tx_id:{txid:14397115}, scn:{branch:0, seq:2120}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397115}, tx_desc:{this:0x7f7ae6ac2ce0, tx_id:{txid:14397115}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621023717, commit_ts:-1, finish_ts:-1, timeout_us:29999991, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621022073, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:2123}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.024753] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397115}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:2123}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.024764] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=10][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant local data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397115} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.024783] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=18][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.024793] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 17, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206C6F63616C2064617461, cstr: tenant local data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36f, hex: 9ECFA11A80300600, int: 1742176621023134}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c377, hex: 9ECFA11A80300600, int: 1742176621023134}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37f, hex: 0040E10100000000, int: 31539200}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c387, hex: 0040E10100000000, int: 31539200}]}) [2025-03-17 09:57:01.024862] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=45][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.025185] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF93C-0-0] [lt=49] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2707}, to_seq_no={branch:0, seq:2121}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397115} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.033928] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF940-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.033954] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF940-0-0] [lt=24][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.033963] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF940-0-0] [lt=6][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 1CF5A11A80300600, int: 1742176621032732}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 1CF5A11A80300600, int: 1742176621032732}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:01.034051] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF940-0-0] [lt=41][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.034314] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117878][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF940-0-0] [lt=49] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2128}, to_seq_no={branch:0, seq:1665}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397117} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.060362] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:01.063096] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463534861}], scn_array=[cnt:1, {val:1742176621035464000, v:0}], curr_size=551, accum_size=3939, time_cost=27367) [2025-03-17 09:57:01.063640] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:01.063689] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=45] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:01.065123] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF942-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.065157] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF942-0-0] [lt=32][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.065170] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF942-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 4F6DA21A80300600, int: 1742176621063503}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 4F6DA21A80300600, int: 1742176621063503}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:01.065258] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF942-0-0] [lt=40][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.065533] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF942-0-0] [lt=36] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2947}, to_seq_no={branch:0, seq:2130}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397118} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.066601] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=4) [2025-03-17 09:57:01.081500] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=55][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.088975] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416801}], scn_array=[cnt:1, {val:1742176620779531002, v:0}], curr_size=122, accum_size=967, time_cost=28606) [2025-03-17 09:57:01.089068] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872705078}], scn_array=[cnt:1, {val:1742176620779531002, v:0}], curr_size=122, accum_size=1154, time_cost=28715) [2025-03-17 09:57:01.110740] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=43] tx gc loop thread is running(MTL_ID()=1003) [2025-03-17 09:57:01.110768] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=27] try gc retain ctx [2025-03-17 09:57:01.110774] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=6] try to retry start_working [2025-03-17 09:57:01.110796] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=4] finish update min start scn(max_decided_scn={val:1742176620995947000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176610989531001, v:0}, keep_alive_scn_:{val:1742176615992731001, v:0}, update_ts_:1742176621110475}) [2025-03-17 09:57:01.111469] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [3539435][T1003_TxLoopWor][T1003][Y0-0000000000000000-0-0] [lt=13] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1003, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:57:01.118760] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=23] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:01.125529] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:01.125575] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:10, fail_bw:40889, [delay/us]:prepare:1, schedule:22, submit:1, rt:3344, total:3370, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:01.125594] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:12288 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4664, iops:36, bw:167647, [delay/us]:prepare:4, schedule:20, submit:358, rt:21359, total:21742) [2025-03-17 09:57:01.125605] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=1, ref_cnt=4, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31964384, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=10, iops=10, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:01.125651] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=42] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:10, avg_bw:40888, [delay/us]: prepare:1, schedule:22, submit:1, device:3344, total:3370) [2025-03-17 09:57:01.125668] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4665, avg_iops:36, avg_bw:167644, [delay/us]: prepare:4, schedule:20, submit:358, device:21359, total:21742) [2025-03-17 09:57:01.125680] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:57:01.125713] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32710, [delay/us]:prepare:1, schedule:35, submit:2, rt:9913, total:9952, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:01.125728] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4608, iops:16, bw:73599, [delay/us]:prepare:7, schedule:19, submit:21, rt:20519, total:20568) [2025-03-17 09:57:01.125738] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:01.125776] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=35] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32710, [delay/us]: prepare:1, schedule:35, submit:2, device:9913, total:9952) [2025-03-17 09:57:01.125793] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4608, avg_iops:16, avg_bw:73597, [delay/us]: prepare:7, schedule:19, submit:21, device:20519, total:20568) [2025-03-17 09:57:01.125804] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:57:01.125827] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8177, [delay/us]:prepare:14, schedule:32, submit:1, rt:25310, total:25358) [2025-03-17 09:57:01.125855] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4300, iops:20, bw:85863, [delay/us]:prepare:7, schedule:20, submit:914, rt:17299, total:18242) [2025-03-17 09:57:01.125869] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:01.125906] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=34] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8177, [delay/us]: prepare:14, schedule:32, submit:1, device:25310, total:25358) [2025-03-17 09:57:01.125921] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4301, avg_iops:20, avg_bw:85859, [delay/us]: prepare:7, schedule:20, submit:914, device:17299, total:18242) [2025-03-17 09:57:01.125934] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:57:01.125959] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4088, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:9829, total:9837) [2025-03-17 09:57:01.125982] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:01.126007] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4088, [delay/us]: prepare:0, schedule:0, submit:600000000, device:9829, total:9837) [2025-03-17 09:57:01.126025] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:57:01.126039] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:01.126054] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126062] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126070] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126077] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126085] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126093] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126100] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126109] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:01.126116] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126124] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126131] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126139] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:01.126160] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS SERVER] net_in=47735kB/s, net_out=786kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:57:01.129755] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=27036672) [2025-03-17 09:57:01.129814] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=60, size_used=0, mem_used=0) [2025-03-17 09:57:01.129878] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=37] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:01.130542] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463535412}], scn_array=[cnt:1, {val:1742176621066450000, v:0}], curr_size=553, accum_size=4492, time_cost=63568) [2025-03-17 09:57:01.132562] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=8][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add19af0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397119} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1088} write_epoch=1, prev=this=0x7f79add082b8 trans_version={val:1742176560044288000, v:0} scn={val:1742176560044288000, v:0} tx_id={txid:14397051} prev=0x7f79add08108 next=(nil) modify_count=3589 acc_checksum=2256531867 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:3209} write_epoch=3, snapshot_version={val:1742176621131734000, v:0}, *this={this=0x7f7778ffe730 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add082b8 latest_compact_node=0x7f78069c7970 max_trans_version={val:1742176560044288000, v:0} max_trans_id=14397051 max_elr_trans_version={val:1742176560044288000, v:0} max_elr_trans_id=14397051 latest_compact_ts=1742176499252827 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176560044288000, v:0} min_modify_scn={val:1742067222497629000, v:0}}) [2025-03-17 09:57:01.132602] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=39][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add19af0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397119} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1088} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176651031876, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176651031876, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621131734000, v:0}, tx_id:{txid:14397119}, scn:{branch:0, seq:1085}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397119}, tx_desc:{this:0x7f7ae6a49660, tx_id:{txid:14397119}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621131734, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621130663, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1088}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.132660] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=57][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397119}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1088}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.132671] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=11][errcode=0] primary key duplication conflict(key={"BIGINT":1002},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant slog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397119} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.132693] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=20][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.132703] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=8][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: E477A31A80300600, int: 1742176621131748}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: E477A31A80300600, int: 1742176621131748}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 00706C0400000000, int: 74215424}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 00706C0400000000, int: 74215424}]}) [2025-03-17 09:57:01.132768] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=37][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.133032] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF944-0-0] [lt=34] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2157}, to_seq_no={branch:0, seq:1086}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397119} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.158817] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=28] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:57:01.159159] INFO [STORAGE.TRANS] submit_log_ (ob_id_service.cpp:158) [117595][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=26] submit log success(service_type_=0, last_id=1742176637787617000, limited_id=1742176637787617000) [2025-03-17 09:57:01.159178] INFO [STORAGE.TRANS] submit_log_with_lock_ (ob_id_service.cpp:65) [117595][TsMgr][T1004][Y0-0000000000000000-0-0] [lt=19] submit log with lock success(service_type_=0, last_id=1742176637787617000, limited_id=1742176637787617000) [2025-03-17 09:57:01.173754] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859708865}], scn_array=[cnt:1, {val:1742176620995947001, v:0}], curr_size=113, accum_size=3762, time_cost=62174) [2025-03-17 09:57:01.175073] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=28] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:57:01.175636] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463535965}], scn_array=[cnt:1, {val:1742176621134888000, v:0}], curr_size=847, accum_size=5339, time_cost=40591) [2025-03-17 09:57:01.175675] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872705200}], scn_array=[cnt:1, {val:1742176621158150001, v:0}], curr_size=128, accum_size=1282, time_cost=16515) [2025-03-17 09:57:01.175775] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162416923}], scn_array=[cnt:1, {val:1742176621158150000, v:0}], curr_size=122, accum_size=1089, time_cost=15358) [2025-03-17 09:57:01.175764] INFO [STORAGE] update_id_meta (ob_ls_meta.cpp:846) [3539271][T1004_ApplySrv1][T1004][YB420A0B093E-00062FB8DBD9BBB0-0-0] [lt=11] update id meta(ret=0, service_type=0, limited_id=1742176637787617000, latest_scn={val:1742176621158150001, v:0}, *this={tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176617654369002, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176637787617000, latest_log_ts:{val:1742176621158150001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}) [2025-03-17 09:57:01.175839] INFO [STORAGE.TRANS] handle_submit_callback (ob_id_service.cpp:190) [3539271][T1004_ApplySrv1][T1004][YB420A0B093E-00062FB8DBD9BBB0-0-0] [lt=74] handle submit callback(service_type_=0, success=true, used_ts=16680, log_ts={val:1742176621158150001, v:0}, limited_id=1742176637787617000) [2025-03-17 09:57:01.175850] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:564) [3539271][T1004_ApplySrv1][T1004][YB420A0B093E-00062FB8DBD9BBB0-0-0] [lt=11] timestamp service handle log callback(ret=0, limited_id=0, log_ts={val:18446744073709551615, v:3}) [2025-03-17 09:57:01.175862] INFO [STORAGE.TRANS] on_success (ob_id_service.cpp:602) [3539271][T1004_ApplySrv1][T1004][YB420A0B093E-00062FB8DBD9BBB0-0-0] [lt=11] ObPresistIDLogCb on success(ret=0, ret="OB_SUCCESS", *this={log_ts:{val:18446744073709551615, v:3}, log_buf:"", pos:0, id_srv_type:-1, limited_id:0}) [2025-03-17 09:57:01.177739] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=7][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add19f30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397120} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1510} write_epoch=1, prev=this=0x7f778764b640 trans_version={val:1742174306987513000, v:0} scn={val:1742174306987513000, v:0} tx_id={txid:14394511} prev=0x7f778764b490 next=(nil) modify_count=21 acc_checksum=3843463606 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4298} write_epoch=3, snapshot_version={val:1742176621176153000, v:0}, *this={this=0x7f7778ffeb90 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x7f778764b640 latest_compact_node=0x7f7763e73250 max_trans_version={val:1742176560049607000, v:0} max_trans_id=14397052 max_elr_trans_version={val:1742176560049607000, v:0} max_elr_trans_id=14397052 latest_compact_ts=1742148340029131 last_compact_cnt=17 total_trans_node_cnt=22 max_modify_scn={val:1742174306987513000, v:0} min_modify_scn={val:1742086112302897000, v:0}}) [2025-03-17 09:57:01.177783] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=43][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add19f30 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397120} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1510} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176651077060, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176651077060, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621176153000, v:0}, tx_id:{txid:14397120}, scn:{branch:0, seq:1109}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397120}, tx_desc:{this:0x7f7ae6a964a0, tx_id:{txid:14397120}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621177221, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621175712, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1510}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.177869] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=85][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397120}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1510}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.177885] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":1005},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant slog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397120} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.177916] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=29][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.177931] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: A123A41A80300600, int: 1742176621175713}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: A123A41A80300600, int: 1742176621175713}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 00B0500000000000, int: 5287936}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 00B0500000000000, int: 5287936}]}) [2025-03-17 09:57:01.178022] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=55][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.178388] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF946-0-0] [lt=49] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2572}, to_seq_no={branch:0, seq:1110}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397120} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.181534] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=31][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.181969] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=26] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:57:01.182606] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539188][T1004_Occam][T1004][YB420A0B093E-00062FB86F654125-0-0] [lt=21] ====== tenant freeze timer task ====== [2025-03-17 09:57:01.187776] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=11][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add1a1a8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397121} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2520} write_epoch=1, prev=this=0x7f79add08970 trans_version={val:1742176560057056000, v:0} scn={val:1742176560057056000, v:0} tx_id={txid:14397053} prev=0x7f79add087c0 next=(nil) modify_count=3589 acc_checksum=2819382034 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4273} write_epoch=3, snapshot_version={val:1742176621186803000, v:0}, *this={this=0x7f7778ffee28 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=2 list_head=0x7f79add08970 latest_compact_node=0x7f79adcf6960 max_trans_version={val:1742176560057056000, v:0} max_trans_id=14397053 max_elr_trans_version={val:1742176560057056000, v:0} max_elr_trans_id=14397053 latest_compact_ts=1742176499313783 last_compact_cnt=3587 total_trans_node_cnt=3590 max_modify_scn={val:1742176560057056000, v:0} min_modify_scn={val:1742067222520977000, v:0}}) [2025-03-17 09:57:01.187817] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=41][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add1a1a8 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397121} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2520} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176651087138, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176651087138, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621186803000, v:0}, tx_id:{txid:14397121}, scn:{branch:0, seq:2517}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397121}, tx_desc:{this:0x7f7ae6b97440, tx_id:{txid:14397121}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621186803, commit_ts:-1, finish_ts:-1, timeout_us:29999991, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621184287, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:2520}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.187874] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=56][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397121}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:2520}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.187887] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=12][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant slog data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397121} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.187909] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=21][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.187921] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420736C6F672064617461, cstr: tenant slog data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 6B4DA41A80300600, int: 1742176621186411}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 6B4DA41A80300600, int: 1742176621186411}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0020940500000000, int: 93593600}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0020940500000000, int: 93593600}]}) [2025-03-17 09:57:01.187986] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=38][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.188302] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF948-0-0] [lt=28] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3577}, to_seq_no={branch:0, seq:2518}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397121} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.195276] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=32] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107785, offset=57287706}, end_cursor:ObLogCursor{file_id=5, log_id=1107786, offset=57288012}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b176fc050, len:3046, log_data_len:306, seq:1107785, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:01.195479] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=22] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:57:01.195516] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176621158150001, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176621158150001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:01.195663] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=16] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:57:01.196353] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=16] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:57:01.196415] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=17] inner load succ(latest_freeze_info={frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615679197000, v:0}, frozen_statuses_.count():8, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}]}) [2025-03-17 09:57:01.196448] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:719) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=31] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1742176615679197000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1004, freeze_info:{latest_snapshot_gc_scn:{val:1742176615679197000, v:0}, frozen_statuses_.count():8, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}]}}) [2025-03-17 09:57:01.196691] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=8238) [2025-03-17 09:57:01.197252] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=8][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:01.197541] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=18] get number(ret=0, service_type_=0, range=1, base_id=1742176621196670000, start_id=1742176621196670000, end_id=1742176621196670001) [2025-03-17 09:57:01.206404] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=7] PNIO [ratelimit] time: 1742176621206402, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007627, add_bytes: 0 [2025-03-17 09:57:01.207474] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=42] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40300924108, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:01.215788] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463538799}], scn_array=[cnt:1, {val:1742176621199860000, v:0}], curr_size=578, accum_size=7904, time_cost=15769) [2025-03-17 09:57:01.216630] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859710154}], scn_array=[cnt:1, {val:1742176621196670000, v:0}], curr_size=1132, accum_size=6070, time_cost=16020) [2025-03-17 09:57:01.216765] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539374][T1003_Occam][T1003][YB420A0B093E-00062FB804154126-0-0] [lt=33] ====== tenant freeze timer task ====== [2025-03-17 09:57:01.218605] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=7][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.218655] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=48][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.218669] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: E903000000000000, int: 1001}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: 03C6A41A80300600, int: 1742176621217283}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: 03C6A41A80300600, int: 1742176621217283}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:01.218797] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=63][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.219166] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=42] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3855}, to_seq_no={branch:0, seq:2123}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397123} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.219221] INFO [STORAGE.TRANS] statistics_ (ob_trans_rpc.cpp:678) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF94C-0-0] [lt=35] rpc statistics(total_trans_msg_count=63, total_batch_msg_count=0) [2025-03-17 09:57:01.233069] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=34] Slow write(duration=16220, write_len=4096) [2025-03-17 09:57:01.233096] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=26] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107786, offset=57288012}, end_cursor:ObLogCursor{file_id=5, log_id=1107787, offset=57288309}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17724050, len:2740, log_data_len:297, seq:1107786, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:01.233297] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:57:01.233336] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731C-0-0] [lt=29] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176621158150000, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176621158150000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:01.246567] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:01.246592] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:01.246852] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176620157204000, replayable_scn:1742176620157204000, readable_scn:1742176620157204000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176621196670000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:01.246888] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176620157204000, replayable_scn:1742176620157204000, readable_scn:1742176620157204000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176621196670000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:01.246906] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176620157204000, replayable_scn:1742176620157204000, readable_scn:1742176620157204000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176621196670000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:01.247994] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFB7-0-0] [lt=7][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:01.248033] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:01.248041] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:57:01.259700] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463539377}], scn_array=[cnt:1, {val:1742176621220085000, v:0}], curr_size=553, accum_size=8457, time_cost=38942) [2025-03-17 09:57:01.262781] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94E-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.262816] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94E-0-0] [lt=32][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.262829] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94E-0-0] [lt=10][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EA03000000000000, int: 1002}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: CF6FA51A80300600, int: 1742176621260751}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: CF6FA51A80300600, int: 1742176621260751}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0060400000000000, int: 4218880}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0060400000000000, int: 4218880}]}) [2025-03-17 09:57:01.262939] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF94E-0-0] [lt=56][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.263253] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF94E-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1961}, to_seq_no={branch:0, seq:899}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397124} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.266701] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=3) [2025-03-17 09:57:01.281602] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.282653] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859711286}], scn_array=[cnt:1, {val:1742176621234020000, v:0}], curr_size=749, accum_size=6819, time_cost=47609) [2025-03-17 09:57:01.282771] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19827, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:57:01.282796] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162417045}], scn_array=[cnt:1, {val:1742176621158150001, v:0}], curr_size=122, accum_size=1211, time_cost=22267) [2025-03-17 09:57:01.282816] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1211) [2025-03-17 09:57:01.282832] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:57:01.282853] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1211) [2025-03-17 09:57:01.282891] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=6, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:57:01.282909] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=17] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19927, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:57:01.282923] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:57:01.283178] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872705328}], scn_array=[cnt:1, {val:1742176621158150002, v:0}], curr_size=122, accum_size=1404, time_cost=22691) [2025-03-17 09:57:01.284196] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=29] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c80b8, ref_cnt_:1, raw_buf_:0x7f7ac6b6e170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78625-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621283499, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:655, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:01.284257] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=59][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.284285] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=26][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.284306] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:01.284315] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c80b8, ref_cnt_:1, raw_buf_:0x7f7ac6b6e170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78625-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621283499, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:655, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:01.284343] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=20][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c80b8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78625-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621283499, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:655, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:-1, end_used:1742176621284337}}, part_id:-1}) [2025-03-17 09:57:01.284381] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=17][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621283499, enqueue_used:2, dequeue_used:20, submit_used:2, return_used:655, callback_enqueue_used:3, callback_dequeue_used:13, callback_finish_used:177, end_used:-34}}) [2025-03-17 09:57:01.284411] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=29][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:01.284430] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=8][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:01.286819] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.286849] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=30][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:01.286856] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7ac862f510, src_buf_size=295) [2025-03-17 09:57:01.286863] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9b70, ref_cnt_:1, raw_buf_:0x7f7ac862e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78625-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621284440, enqueue_used:1, dequeue_used:28, submit_used:1, return_used:2337, callback_enqueue_used:3, callback_dequeue_used:6, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:01.286915] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=45][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9b70, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78625-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621284440, enqueue_used:1, dequeue_used:28, submit_used:1, return_used:2337, callback_enqueue_used:3, callback_dequeue_used:6, callback_finish_used:-1, end_used:1742176621286911}}, part_id:-1}) [2025-03-17 09:57:01.286921] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=47][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463539930}], scn_array=[cnt:1, {val:1742176621263789000, v:0}], curr_size=553, accum_size=9010, time_cost=21953) [2025-03-17 09:57:01.286942] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176621284440, enqueue_used:1, dequeue_used:28, submit_used:1, return_used:2337, callback_enqueue_used:3, callback_dequeue_used:6, callback_finish_used:119, end_used:-24}}) [2025-03-17 09:57:01.286961] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=18][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:01.286968] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=5][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a41072080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:01.286988] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=20][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a41072080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:01.287006] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=14][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:01.287020] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=13][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cb697d0}) [2025-03-17 09:57:01.287039] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=19][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cb6a578, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:0, rowkey_end_idx:0, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cb6a578, micro_handle:NULL}) [2025-03-17 09:57:01.287082] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=42][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:57:01.287090] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=8][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a41042a58, local_datums:0x7f7a41042a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:57:01.287299] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=208][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a41042a58, local_datums:0x7f7a41042a58}, datums:[]}) [2025-03-17 09:57:01.287311] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=11][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176621263789000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:57:01.287328] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=16][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7a41041ac0) [2025-03-17 09:57:01.287336] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=7][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:57:01.287344] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=7][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1},{"BIGINT":1};{"BIGINT":1},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176651182497, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7a410413d0, valid:true, source:2, core:{version:{val:1742176621263789000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176621282920, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7a4103d9a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7a410416a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7a4103d848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7a41041ac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:57:01.287616] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=272][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:01.287622] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:57:01.287626] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=4][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:57:01.287630] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=4][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:57:01.287635] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=4][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:57:01.287639] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=4][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:01.287646] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a41039b50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:01.287657] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=10][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:57:01.287664] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=7][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:57:01.287671] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:57:01.287675] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=4][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:57:01.287683] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a41039b50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:57:01.287692] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=8][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:01.287698] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=6][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:01.287705] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=7][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:57:01.287712] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=7][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1"}) [2025-03-17 09:57:01.287722] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=8][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:01.287750] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78625-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade9b3980, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222517477, assoc_session_id:3222517477, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176621282921, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176621282920, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a2b3ac1f8, id:3222517477, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade9b3980}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.288910] INFO [SHARE] get_balance_job (ob_balance_job_table_operator.cpp:209) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=28] empty balance job(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=select time_to_usec(gmt_create) as start_time, time_to_usec(gmt_modified) as finish_time, * from __all_balance_job) [2025-03-17 09:57:01.289116] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=10][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add1aca0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397125} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1067} write_epoch=1, prev=this=0x7f79adc86ff0 trans_version={val:1742176133543074000, v:0} scn={val:1742176133543074000, v:0} tx_id={txid:14396578} prev=0x7f79adc86e40 next=(nil) modify_count=45 acc_checksum=1224837421 version=1742067188266264 type=0 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:4250} write_epoch=3, snapshot_version={val:1742176621288289000, v:0}, *this={this=0x7f7778fff798 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=4 list_head=0x7f79adc86ff0 latest_compact_node=0x7f78e79b4d20 max_trans_version={val:1742176560117692000, v:0} max_trans_id=14397057 max_elr_trans_version={val:1742176560117692000, v:0} max_elr_trans_id=14397057 latest_compact_ts=1742163998317934 last_compact_cnt=41 total_trans_node_cnt=46 max_modify_scn={val:1742176133543074000, v:0} min_modify_scn={val:1742078732906602000, v:0}}) [2025-03-17 09:57:01.289179] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=62][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add1aca0 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397125} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:1067} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176651188209, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176651188209, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621288289000, v:0}, tx_id:{txid:14397125}, scn:{branch:0, seq:1064}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397125}, tx_desc:{this:0x7f7ae6a9c5c0, tx_id:{txid:14397125}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621288289, commit_ts:-1, finish_ts:-1, timeout_us:29999988, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621287410, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:1067}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.289263] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=83][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397125}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:1067}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.289293] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=30][errcode=0] primary key duplication conflict(key={"BIGINT":1005},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant meta data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397125} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.289328] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=33][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.289343] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: ED03000000000000, int: 1005}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: F3D7A51A80300600, int: 1742176621287411}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: F3D7A51A80300600, int: 1742176621287411}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 00A0BA0000000000, int: 12230656}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 00A0BA0000000000, int: 12230656}]}) [2025-03-17 09:57:01.289420] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=47][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.289806] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF950-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2134}, to_seq_no={branch:0, seq:1065}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397125} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.289875] INFO [BALANCE] build (ob_transfer_partition_task.cpp:120) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=26] [TRANSFER_PARTITION] no transfer partition task(tenant_id_=1004) [2025-03-17 09:57:01.289892] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1010) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=15] no transfer partition job [2025-03-17 09:57:01.289902] INFO [BALANCE] transfer_partition_ (ob_tenant_balance_service.cpp:1040) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=8] [TENANT_BALANCE] finish transfer partition(ret=0, ret="OB_SUCCESS", job_cnt=0, enable transfer=true) [2025-03-17 09:57:01.291349] INFO [BALANCE] gather_ls_status_stat (ob_tenant_balance_service.cpp:284) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=9] [TENANT_BALANCE] LS no need balance(ls_status={tenant_id:1004, ls_id:{id:1}, ls_group_id:0, status:"NORMAL", unit_group_id:0, primary_zone:"zone1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}) [2025-03-17 09:57:01.291376] INFO [BALANCE] ls_balance_ (ob_tenant_balance_service.cpp:493) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=20] [TENANT_BALANCE] finish ls balance(ret=0, ret="OB_SUCCESS", need_ls_balance=false, job={tenant_id:0, job_id:{id:-1}, job_type:{val:-1, job_type:"INVALID"}, job_status:{val:-1, job_status:"INVALID"}, primary_zone_num:0, unit_group_num:0, comment:, balance_strategy:}, tasks=[]) [2025-03-17 09:57:01.292762] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=16][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:01.292838] INFO [BALANCE] do_work (ob_tenant_balance_service.cpp:157) [3539183][T1004_TBalance][T1004][YB420A0B093E-00062FB7FD0623CF-0-0] [lt=31] [TENANT_BALANCE] finish one round(ret=0, ret="OB_SUCCESS", tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, job_cnt=0, primary_zone_num_=1, unit_group_array_=[{unit_group_id:1004, status:0}], ls_array_=[{tenant_id:1004, ls_id:{id:1001}, ls_group_id:1001, status:"NORMAL", unit_group_id:1004, primary_zone:"zone1", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}], idle_time_us=10000000, last_partition_balance_time=1742172358002579, last_statistic_bg_stat_time=1742009087220102, last_statistic_schema_version=1742008591889280, last_statistic_max_transfer_task_id={id:-1}, enable_rebalance=true, enable_transfer=true) [2025-03-17 09:57:01.295398] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=47] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1003, server_version={version:{val:1742176620995947000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=299442, in_cluster_service=true, cluster_version={val:1742176620995947000, v:0}, min_cluster_version={val:1742176620995947000, v:0}, max_cluster_version={val:1742176620995947000, v:0}, get_cluster_version_err=0, cluster_version_delta=299442, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=5762177, succ_cluster_heartbeat_count=8, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176264250519000, v:0}, local_cluster_delta=357044870, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:57:01.309765] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=5] PNIO [ratelimit] time: 1742176621309763, bytes: 2535063852475, bw: 52.868492 MB/s, add_ts: 1002856, add_bytes: 55594959 [2025-03-17 09:57:01.324236] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7B-0-0] [lt=28] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:57:01.324264] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7B-0-0] [lt=27] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:57:01.330600] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463540483}], scn_array=[cnt:1, {val:1742176621291500000, v:0}], curr_size=553, accum_size=9563, time_cost=38034) [2025-03-17 09:57:01.330686] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=40, avg_log_gen_to_freeze_time=4936, avg_log_gen_to_submit_time=4947, avg_log_submit_to_flush_time=22268, avg_log_submit_to_slide_time=22288) [2025-03-17 09:57:01.330737] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=23846, hit_cnt=41, read_cnt=41, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:01.333502] WDIAG [STORAGE.TRANS] mvcc_sanity_check_ (ob_mvcc_row.cpp:956) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=41][errcode=-5024] find double insert node(ret=-5024, node=this=0x7f79add1af18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397126} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2760} write_epoch=1, prev=this=0x7f7763eae2f0 trans_version={val:1742149863388735000, v:0} scn={val:1742149863388735000, v:0} tx_id={txid:13986973} prev=0x7f786a0b90c0 next=(nil) modify_count=11 acc_checksum=3540803747 version=1742067188266264 type=1 flag=4 snapshot_barrier=1742176620146518000 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:18} write_epoch=0, snapshot_version={val:1742176621332020000, v:0}, *this={this=0x7f7778fffa30 latch_=locked flag=11 first_dml=DELETE last_dml=INSERT update_since_compact=0 list_head=0x7f7763eae2f0 latest_compact_node=0x7f7763eae2f0 max_trans_version={val:1742176560136256000, v:0} max_trans_id=14397058 max_elr_trans_version={val:1742176560136256000, v:0} max_elr_trans_id=14397058 latest_compact_ts=1742149863425500 last_compact_cnt=11 total_trans_node_cnt=12 max_modify_scn={val:1742149863388735000, v:0} min_modify_scn={val:1742071728953595000, v:0}}) [2025-03-17 09:57:01.333545] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_row.cpp:1018) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=42][errcode=-5024] mvcc write failed(ret=-5024, node=this=0x7f79add1af18 trans_version={val:4611686018427387903, v:0} scn={val:4611686018427387903, v:0} tx_id={txid:14397126} prev=(nil) next=(nil) modify_count=4294967295 acc_checksum=0 version=1742067188266264 type=0 flag=128 snapshot_barrier=0 snapshot_barrier_flag=0 mtd={dml_flag:3, buf_len:88} seq_no={branch:0, seq:2760} write_epoch=1, ctx={this:0x7f7af54d3658, ls_id:{id:1}, ls:0x7f7ac1de6150, branch:0, timeout:1742176651232741, tablet_id:{id:309}, table_iter:0x7f7af54d1b48, table_version:1741278086776728, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176651232741, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176621332020000, v:0}, tx_id:{txid:14397126}, scn:{branch:0, seq:2129}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:14397126}, tx_desc:{this:0x7f7ae6a0cb20, tx_id:{txid:14397126}, state:1, addr:"192.168.1.63:2882", tenant_id:1, session_id:1, assoc_session_id:0, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:2, alloc_ts:-1, active_ts:1742176621333106, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621330349, flags_.SHADOW:true, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, tx_ctx:0x7f7b0b745cd0, mem_ctx:0x7f7b0b747250, tx_scn:{branch:0, seq:2760}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:1, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:01.333604] WDIAG [STORAGE.TRANS] mvcc_write (ob_mvcc_engine.cpp:337) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=58][errcode=-5024] mvcc write failed(ret=-5024, arg={tx_id:{txid:14397126}, data:0x7f7af54d0508, old_row:null, modify_count:4294967295, acc_checksum:0, memstore_version:1742067188266264, seq_no:{branch:0, seq:2760}, write_epoch:1, scn:{val:4611686018427387903, v:0}, column_cnt:9}) [2025-03-17 09:57:01.333620] WDIAG [STORAGE.TRANS] on_key_duplication_retry (ob_memtable_context.cpp:329) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=15][errcode=0] primary key duplication conflict(key={"BIGINT":1006},{"VARCHAR":"192.168.1.63", collation:"utf8mb4_general_ci"},{"BIGINT":2882},{"BIGINT":1},{"VARCHAR":"tenant meta data", collation:"utf8mb4_general_ci"}, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=1 trans_id={txid:14397126} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=1 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=0, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,1,0,0,1,0,0,0)|]}}) [2025-03-17 09:57:01.333650] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=30][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.333681] INFO [LIB] stat (utility.h:1210) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=16, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:57:01.333664] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=11][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: EE03000000000000, int: 1006}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 16, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E74206D6574612064617461, cstr: tenant meta data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36e, hex: D483A61A80300600, int: 1742176621331412}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c376, hex: D483A61A80300600, int: 1742176621331412}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37e, hex: 0030E60000000000, int: 15085568}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c386, hex: 0030E60000000000, int: 15085568}]}) [2025-03-17 09:57:01.333756] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=67][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.334019] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF952-0-0] [lt=17] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3192}, to_seq_no={branch:0, seq:2130}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397126} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.339098] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=23] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:57:01.346089] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=34] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:01.347976] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=27] gc stale ls task succ [2025-03-17 09:57:01.350318] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.350354] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=34][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.350370] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=16][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:01.350377] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7380, ref_cnt_:1, raw_buf_:0x7f7aa4274150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63406-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176621348653, enqueue_used:3, dequeue_used:17, submit_used:1, return_used:1627, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.350416] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=28][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7380, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63406-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176621348653, enqueue_used:3, dequeue_used:17, submit_used:1, return_used:1627, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176621350407}}, part_id:-1}) [2025-03-17 09:57:01.350818] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=28][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.350847] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=28][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:01.350854] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d1c315a, src_buf_size=1611) [2025-03-17 09:57:01.350861] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7a40, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63406-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176621350485, enqueue_used:2, dequeue_used:16, submit_used:0, return_used:290, callback_enqueue_used:1, callback_dequeue_used:22, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.350898] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=30][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7a40, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63406-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176621350485, enqueue_used:2, dequeue_used:16, submit_used:0, return_used:290, callback_enqueue_used:1, callback_dequeue_used:22, callback_finish_used:-1, end_used:1742176621350892}}, part_id:-1}) [2025-03-17 09:57:01.351048] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=1] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222521041, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf39f5c0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:01.351109] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=45] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3204}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634369} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:01.351163] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=33] end stmt(ret=0, tx_id=9634369, plain_select=false, stmt_type=2, savepoint={branch:0, seq:5}, tx_desc={this:0x7f7acf39f5c0, tx_id:{txid:9634369}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222521041, assoc_session_id:3222521041, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176621348073, active_ts:1742176621348073, commit_ts:-1, finish_ts:-1, timeout_us:29999988, lock_timeout_us:-1, expire_ts:1742176651348061, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465462744193, first_scn:{branch:0, seq:4}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176621347008}], exec_info_reap_ts:1742176621347007, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621347003, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222521041, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf39f5c0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.351330] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634369}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176651348061, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621348073}{self_ls_id:{id:1}, session_id:3222521041, associated_session_id:3222521041, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634369}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465462744193, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621348073, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634369} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.351452] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=81] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634369}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176651348061, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621348073}{self_ls_id:{id:1}, session_id:3222521041, associated_session_id:3222521041, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634369}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465462744193, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621348073, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634369} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.351535] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=76] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634369}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176621351326, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:01.351556] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=18] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf39f5c0, tx_id:{txid:9634369}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222521041, assoc_session_id:3222521041, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176621348073, active_ts:1742176621348073, commit_ts:-1, finish_ts:1742176621351263, timeout_us:29999988, lock_timeout_us:-1, expire_ts:1742176651348061, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465462744193, first_scn:{branch:0, seq:4}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176621347008}], exec_info_reap_ts:1742176621347007, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621347003, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:01.351662] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63406-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:01.357780] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=30] ====== check clog disk timer task ====== [2025-03-17 09:57:01.357812] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=27] get_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:57:01.361333] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463541036}], scn_array=[cnt:1, {val:1742176621335271000, v:0}], curr_size=553, accum_size=10116, time_cost=25531) [2025-03-17 09:57:01.363684] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF954-0-0] [lt=10][errcode=-5024] internal lock row failed(ret=-5024, ret="OB_ERR_PRIMARY_KEY_DUPLICATE") [2025-03-17 09:57:01.363722] WDIAG [STORAGE] insert_row_without_rowkey_check (ob_tablet.cpp:4652) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF954-0-0] [lt=37][errcode=-5024] fail to set memtable(ret=-5024) [2025-03-17 09:57:01.363734] WDIAG [STORAGE] insert_row_to_tablet (ob_ls_tablet_service.cpp:5675) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF954-0-0] [lt=9][errcode=-5024] failed to write table row(ret=-5024, table id=309, col_descs=[cnt:9, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], datum_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:9, datum_buffer:{capacity:16, datums:0x7f781dc0c690, local_datums:0x7f781dc0c690}, datums:[col_id=0:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c33c, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=1:{len: 10, flag: 0, null: 0, ptr: 0x7f781dc0c344, hex: 31302E31312E392E3633, cstr: 192.168.1.63}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c34e, hex: 420B000000000000, int: 2882}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c356, hex: 0100000000000000, int: 1, num_digit0: 0}, col_id=4:{len: 15, flag: 0, null: 0, ptr: 0x7f781dc0c35e, hex: 74656E616E7420746D702064617461, cstr: tenant tmp data}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c36d, hex: 57FDA61A80300600, int: 1742176621362519}, col_id=6:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c375, hex: 57FDA61A80300600, int: 1742176621362519}, col_id=7:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c37d, hex: 0000000000000000, int: 0}, col_id=8:{len: 8, flag: 0, null: 0, ptr: 0x7f781dc0c385, hex: 0000000000000000, int: 0}]}) [2025-03-17 09:57:01.363841] WDIAG [STORAGE] insert_row (ob_ls_tablet_service.cpp:3334) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB8CE6BF954-0-0] [lt=56][errcode=-5024] failed to write row(ret=-5024) [2025-03-17 09:57:01.364198] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [117887][T1_L0_G0][T1][YB420A0B093F-00062FB8CE6BF954-0-0] [lt=24] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1561}, to_seq_no={branch:0, seq:494}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741278086776728 max_table_version=1741278086776728 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:14397127} ls_id=1 row_callback[alloc:0, free:0, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=0 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,1,1,1,0)|]}}) [2025-03-17 09:57:01.381666] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.384628] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463541589}], scn_array=[cnt:1, {val:1742176621348073000, v:0}], curr_size=587, accum_size=10703, time_cost=23172) [2025-03-17 09:57:01.384733] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=18] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=41, avg_log_batch_cnt=1, total_group_log_size=24397, avg_group_log_size=595) [2025-03-17 09:57:01.395480] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:690) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=53] no need to update min/max version(new_version={val:1742176620995947000, v:0}, min_version={val:1742176620995947000, v:0}, max_version={val:1742176620995947000, v:0}) [2025-03-17 09:57:01.404022] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463542176}], scn_array=[cnt:1, {val:1742176621365156000, v:0}], curr_size=549, accum_size=11252, time_cost=19238) [2025-03-17 09:57:01.407796] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=6] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40298826956, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:01.415075] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.415171] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.415821] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.416093] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.416092] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=10] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.416187] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=31] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.416729] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.432882] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=26] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:01.439911] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=21] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:57:01.440654] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=33] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:57:01.440720] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=25] inner load succ(latest_freeze_info={frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615071995000, v:0}, frozen_statuses_.count():3, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629601710007000, v:0}, schema_version:1741597433156656, data_version:17180067072}, {frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}]}) [2025-03-17 09:57:01.444615] INFO [SQL.DTL] runTimerTask (ob_dtl_interm_result_manager.cpp:685) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=17] clear dtl interm result cost(us)(clear_cost=6, ret=0, gc_.expire_keys_.count()=0, dump count=0, clean count=0) [2025-03-17 09:57:01.449598] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=12] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:01.451242] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.451279] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=36][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.451297] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:01.451311] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=13][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5e20, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63407-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176621450808, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:393, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.451362] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=40][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c5e20, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63407-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176621450808, enqueue_used:2, dequeue_used:23, submit_used:1, return_used:393, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176621451353}}, part_id:-1}) [2025-03-17 09:57:01.451376] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=41][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:01.452510] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.452545] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=35][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:01.452553] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d00f15a, src_buf_size=1611) [2025-03-17 09:57:01.452560] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c83d0, ref_cnt_:1, raw_buf_:0x7f791d00e150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63407-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176621451414, enqueue_used:0, dequeue_used:25, submit_used:1, return_used:1054, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.452597] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=26][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c83d0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63407-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4262080, time_log:{begin_ts:1742176621451414, enqueue_used:0, dequeue_used:25, submit_used:1, return_used:1054, callback_enqueue_used:3, callback_dequeue_used:10, callback_finish_used:-1, end_used:1742176621452588}}, part_id:-1}) [2025-03-17 09:57:01.452724] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=1] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222521415, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2d5040}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:01.452765] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3169}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634372} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:01.452791] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=22] end stmt(ret=0, tx_id=9634372, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2d5040, tx_id:{txid:9634372}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222521415, assoc_session_id:3222521415, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176621449335, active_ts:1742176621449335, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176651449329, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465564950718, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621448274}], exec_info_reap_ts:1742176621448273, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621448270, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222521415, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2d5040}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.452940] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634372}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176651449329, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621450361}{self_ls_id:{id:1}, session_id:3222521415, associated_session_id:3222521415, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634372}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465564950718, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621450361, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634372} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.453014] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=61] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634372}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176651449329, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621450361}{self_ls_id:{id:1}, session_id:3222521415, associated_session_id:3222521415, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634372}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465564950718, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621450361, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634372} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.453071] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=51] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634372}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176621452934, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:01.453093] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=19] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2d5040, tx_id:{txid:9634372}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222521415, assoc_session_id:3222521415, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176621449335, active_ts:1742176621449335, commit_ts:-1, finish_ts:1742176621452505, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176651449329, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465564950718, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621448274}], exec_info_reap_ts:1742176621448273, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621448270, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:01.453198] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63407-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:01.465988] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=33] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}]) [2025-03-17 09:57:01.466820] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=2) [2025-03-17 09:57:01.474523] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872705572}], scn_array=[cnt:1, {val:1742176621158150004, v:0}], curr_size=122, accum_size=1648, time_cost=13892) [2025-03-17 09:57:01.481737] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.481734] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=26] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:01.481751] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=22] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:01.481770] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=30] temporary file module use no blocks [2025-03-17 09:57:01.481774] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=20] temporary file module use no blocks [2025-03-17 09:57:01.481782] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:01.481779] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=8] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:01.481792] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=9] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:01.481792] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:01.481807] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=22] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:01.481805] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863116373535, flush_io_finished_ret_:0, flush_io_finished_round_:580429, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:01.481806] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863116373500, flush_io_finished_ret_:0, flush_io_finished_round_:859379, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:01.481829] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=19] temporary file module use no blocks [2025-03-17 09:57:01.481835] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=5] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:01.481848] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:01.481858] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=9] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863116373597, flush_io_finished_ret_:0, flush_io_finished_round_:580430, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:01.512111] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=11] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=0, elapse_time=0, size_used=196608, mem_used=370194432) [2025-03-17 09:57:01.512189] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=14] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:57:01.530520] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0075-0-0] [lt=20][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:01.530567] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0075-0-0] [lt=45][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.530621] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0075-0-0] [lt=47][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:01.531050] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0075-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace89e620, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222521602, assoc_session_id:3222521602, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176621529153, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176621529152, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222521602, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace89e620}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.535287] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:257) [117825][T1_ArcSender][T1][YB420A0B093E-00062FB7EE954125-0-0] [lt=33][errcode=-4018] try free send task failed(ret=-4018) [2025-03-17 09:57:01.537419] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859712623}], scn_array=[cnt:1, {val:1742176621496127000, v:0}], curr_size=588, accum_size=7995, time_cost=40800) [2025-03-17 09:57:01.537482] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=62] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=7995) [2025-03-17 09:57:01.537502] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=17, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:57:01.537561] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=258] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=7, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:57:01.537580] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=18] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=26184, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:57:01.537589] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=3119, avg_log_gen_to_submit_time=3130, avg_log_submit_to_flush_time=23008, avg_log_submit_to_slide_time=23053) [2025-03-17 09:57:01.537611] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=65, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:57:01.537619] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=7995, hit_cnt=14, read_cnt=14, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:01.537829] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [117927][BlackListServic][T1][YB420A0B093E-00062FB7EEF9A097-0-0] [lt=23] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12239, add_timestamp:1742176621537814}) [2025-03-17 09:57:01.537876] INFO [SHARE.LOCATION] vtable_nonblock_renew (ob_vtable_location_service.cpp:446) [117927][BlackListServic][T1][YB420A0B093E-00062FB7EEF9A097-0-0] [lt=26] add_update_task succeed(ret=0, ret="OB_SUCCESS", task={tenant_id:1, table_id:12287, add_timestamp:1742176621537873}) [2025-03-17 09:57:01.537873] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A9-0-0] [lt=36] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12239, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621537855, sql_renew_time:1742176621537855, is_mark_fail:false}]) [2025-03-17 09:57:01.537917] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31A9-0-0] [lt=44] success to process renew task(task={tenant_id:1, table_id:12239, add_timestamp:1742176621537814}, locations=[{table_id:12239, partition_id:12239, partition_cnt:0, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621537855, sql_renew_time:1742176621537855, is_mark_fail:false}]) [2025-03-17 09:57:01.537941] INFO [SHARE.LOCATION] fetch_vtable_location_ (ob_vtable_location_service.cpp:281) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31AA-0-0] [lt=16] fetch virtual table location with tenant(rpc tenant=0, tenant_id=1, table_id=12287, vtable_type={tenant_id:1, type:4}) [2025-03-17 09:57:01.538405] INFO [SHARE.LOCATION] renew_vtable_location_ (ob_vtable_location_service.cpp:207) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31AA-0-0] [lt=10] renew vtable location success(ret=0, ret="OB_SUCCESS", tenant_id=1, table_id=12287, locations=[{table_id:12287, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:01.538451] INFO [SHARE.LOCATION] batch_process_tasks (ob_vtable_location_service.cpp:414) [117550][VTblLocAsyncUp0][T0][YB420A0B093E-00062FB9096E31AA-0-0] [lt=45] success to process renew task(task={tenant_id:1, table_id:12287, add_timestamp:1742176621537873}, locations=[{table_id:12287, partition_id:0, partition_cnt:4, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:1, partition_cnt:4, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:2, partition_cnt:4, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}, {table_id:12287, partition_id:3, partition_cnt:4, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, reserved:0, property:{memstore_percent_:100}}], renew_time:1742176621538393, sql_renew_time:0, is_mark_fail:false}]) [2025-03-17 09:57:01.544160] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1C-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.544217] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1C-0-0] [lt=11][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:01.544746] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1C-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6a90380, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222521097, assoc_session_id:3222521097, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176621543003, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176621541979, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7af2520f58, id:3222521097, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6a90380}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:01.549491] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:261) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1002) [2025-03-17 09:57:01.549521] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.549547] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.549565] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.549645] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] switch tenant fail(tenant_id=1001, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.549667] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.550319] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.550351] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-5150] switch tenant fail(tenant_id=1005, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.550365] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=8][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.550378] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=7][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:01.550655] INFO [STORAGE.TRANS] print_stat_ (ob_black_list.cpp:399) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=6] start to print blacklist info [2025-03-17 09:57:01.550801] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:194) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=16] ls blacklist refresh finish(cost_time=13738) [2025-03-17 09:57:01.563751] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=23] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:01.563809] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=39] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:01.564734] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859713211}], scn_array=[cnt:1, {val:1742176621533386000, v:0}], curr_size=588, accum_size=588, time_cost=27104) [2025-03-17 09:57:01.564980] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=51][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463542847}], scn_array=[cnt:1, {val:1742176621547239000, v:0}], curr_size=587, accum_size=11961, time_cost=16798) [2025-03-17 09:57:01.566694] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=51] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1770) [2025-03-17 09:57:01.566766] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1770) [2025-03-17 09:57:01.566824] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=18] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1770, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:01.581801] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.584702] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=358, pos=299) [2025-03-17 09:57:01.585612] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=30][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=57, len=292, pos=262) [2025-03-17 09:57:01.586021] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=17][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=56, pos=48) [2025-03-17 09:57:01.586597] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=30, len=223, pos=194) [2025-03-17 09:57:01.587196] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=77, len=443, pos=413) [2025-03-17 09:57:01.587740] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=7][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=60, pos=48) [2025-03-17 09:57:01.588316] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=54, len=268, pos=239) [2025-03-17 09:57:01.589072] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571B-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=44, len=264, pos=234) [2025-03-17 09:57:01.602590] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=11] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:01.604251] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=23] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c93d8, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63408-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176621603793, enqueue_used:3, dequeue_used:35, submit_used:1, return_used:406, callback_enqueue_used:3, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:01.604315] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=62][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.604338] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=21][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.604353] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=14][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:01.604359] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c93d8, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63408-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176621603793, enqueue_used:3, dequeue_used:35, submit_used:1, return_used:406, callback_enqueue_used:3, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.604389] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=18][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c93d8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63408-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176621603793, enqueue_used:3, dequeue_used:35, submit_used:1, return_used:406, callback_enqueue_used:3, callback_dequeue_used:8, callback_finish_used:-1, end_used:1742176621604380}}, part_id:-1}) [2025-03-17 09:57:01.604657] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=17][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.604676] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=19][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:01.604683] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f791d1c315a, src_buf_size=1611) [2025-03-17 09:57:01.604690] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c87c0, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63408-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176621604410, enqueue_used:1, dequeue_used:38, submit_used:1, return_used:199, callback_enqueue_used:1, callback_dequeue_used:5, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.604726] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=28][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c87c0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63408-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176621604410, enqueue_used:1, dequeue_used:38, submit_used:1, return_used:199, callback_enqueue_used:1, callback_dequeue_used:5, callback_finish_used:-1, end_used:1742176621604719}}, part_id:-1}) [2025-03-17 09:57:01.604854] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222505034, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf323ee0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:01.604914] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=44] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634375} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:01.604949] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=31] end stmt(ret=0, tx_id=9634375, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf323ee0, tx_id:{txid:9634375}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222505034, assoc_session_id:3222505034, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176621602691, active_ts:1742176621602691, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176651602686, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465717940413, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621601625}], exec_info_reap_ts:1742176621601624, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621601621, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222505034, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf323ee0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.605082] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634375}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176651602686, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621602691}{self_ls_id:{id:1}, session_id:3222505034, associated_session_id:3222505034, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634375}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465717940413, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621602691, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634375} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.605187] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=89] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634375}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176651602686, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621602691}{self_ls_id:{id:1}, session_id:3222505034, associated_session_id:3222505034, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634375}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465717940413, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621602691, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634375} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.605243] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=51] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634375}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176621605078, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:01.605265] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=20] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf323ee0, tx_id:{txid:9634375}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222505034, assoc_session_id:3222505034, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176621602691, active_ts:1742176621602691, commit_ts:-1, finish_ts:1742176621604820, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176651602686, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465717940413, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621601625}], exec_info_reap_ts:1742176621601624, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621601621, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:01.605347] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63408-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:01.608142] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=31] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40290438348, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:01.623019] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=20] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:57:01.629947] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=18] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=27036672) [2025-03-17 09:57:01.630008] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=61, size_used=0, mem_used=0) [2025-03-17 09:57:01.630055] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=22] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:01.635387] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=15] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.635402] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=17] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.635440] INFO [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:263) [117825][T1_ArcSender][T1][YB420A0B093E-00062FB7EE954125-0-0] [lt=30] ObArchiveSender is running(thread_index=0) [2025-03-17 09:57:01.635914] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636072] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636076] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636069] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636133] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636172] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636224] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.636335] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:01.641628] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=41] gc stale ls task succ [2025-03-17 09:57:01.654221] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=26] ====== check clog disk timer task ====== [2025-03-17 09:57:01.654245] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=21] get_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:57:01.659484] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=7] ts response task statistics(alloc_count=16165392, free_count=8082696) [2025-03-17 09:57:01.660762] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1648, avg_group_log_size=164) [2025-03-17 09:57:01.660804] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=38] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1648) [2025-03-17 09:57:01.660839] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:01.660856] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:01.666893] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=22] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=1) [2025-03-17 09:57:01.674201] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=37] gc stale ls task succ [2025-03-17 09:57:01.677373] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=31, pos=15) [2025-03-17 09:57:01.678072] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=29][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=83, len=530, pos=504) [2025-03-17 09:57:01.678607] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=16][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=41, pos=15) [2025-03-17 09:57:01.679033] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=83, len=582, pos=551) [2025-03-17 09:57:01.679471] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=76, len=505, pos=503) [2025-03-17 09:57:01.679908] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=199, pos=171) [2025-03-17 09:57:01.680334] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571D-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=0, len=5, pos=4) [2025-03-17 09:57:01.681082] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:470) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=41] skip defragment task(ret=0, need_defragment=false) [2025-03-17 09:57:01.681103] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:512) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=20] successfully defragment data blocks(ret=0, rewrite_cnt=0, block_used_size_.count()=10) [2025-03-17 09:57:01.681111] INFO [STORAGE.BLKMGR] defragment (ob_shared_macro_block_manager.cpp:530) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=8] successfully defragment data blocks(rewrite_cnt=0) [2025-03-17 09:57:01.681138] INFO [STORAGE] ~ObITabletFilterOp (ob_tablet_pointer.cpp:765) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=25] tablet skip filter destructed(total_cnt=0, skip_cnt=0, not_in_mem_cnt=0, invalid_attr_cnt=0, total_tablet_cnt=15851311, total_skip_cnt=45934, not_in_mem_tablet_cnt=0, invalid_attr_tablet_cnt=0) [2025-03-17 09:57:01.681872] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.697260] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=36] current monitor number(seq_=-1) [2025-03-17 09:57:01.697477] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2747) [117795][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=20] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", [SlidingWindow]={palf_id:1, self:"192.168.1.62:2882", lsn_allocator:{max_log_id:10438969, max_lsn:5463543556, max_scn:{val:1742176621658961000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:5463543556}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:5421600516}}, last_submit_lsn:{lsn:5463543434}, last_submit_end_lsn:{lsn:5463543556}, last_submit_log_id:10438969, last_submit_log_pid:4, max_flushed_lsn:{lsn:5463543434}, max_flushed_end_lsn:{lsn:5463543556}, max_flushed_log_pid:4, committed_end_lsn:{lsn:5463543556}, last_slide_log_id:10438969, last_slide_scn:{val:1742176621658961000, v:0}, last_slide_lsn:{lsn:5463543434}, last_slide_end_lsn:{lsn:5463543556}, last_slide_log_pid:4, last_slide_log_accum_checksum:3020055542, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f7adde5c530}, [StateMgr]={this:0x7f7adde60830, self:"192.168.1.62:2882", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:4}, leader:"192.168.1.62:2882", leader_epoch:2, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:10438970, is_changing_config_with_arb:false, reconfirm_start_time_us:1741318427151411, palf_role_change_cb:0x7f7adde672f0, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"192.168.1.62:2882", alive_paxos_memberlist:1{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}, alive_paxos_replica_num:1, log_ms_meta:{version:3, proposal_id:4, prev:{version:1, config:{config_version:{proposal_id:4, config_seq:9}, log_sync_memberlist:2{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}{server:"192.168.1.63:2882", timestamp:1, flag:0}, log_sync_replica_num:2, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:4, config_seq:10}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:4, prev_lsn:{lsn:64074526}, prev_mode_pid:4}, running_args:{server:{server:"192.168.1.63:2882", timestamp:1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:0, config_version:{proposal_id:3, config_seq:7}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"REMOVE_MEMBER_AND_NUM"}, state:0, checking_barrier:{prev_log_proposal_id:4, prev_lsn:{lsn:64074526}, prev_end_lsn:{lsn:64075557}, prev_mode_pid:4}, reconfig_barrier:{prev_log_proposal_id:4, prev_lsn:{lsn:64074526}, prev_end_lsn:{lsn:64075557}, prev_mode_pid:4}, persistent_config_version:{proposal_id:4, config_seq:10}, ms_ack_list:0, resend_config_version:{proposal_id:4, config_seq:10}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:1741318423649226, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:1741318423649226, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f7adde5dc70}, [ModeMgr]={palf_id:1, self:"192.168.1.62:2882", applied_mode_meta:{version:1, proposal_id:4, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:4, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:4, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:1741318427351574, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:18446744073709551615, v:3}, min_block_id:0, min_block_min_scn:{val:2, v:0}, base_lsn_for_block_gc:{lsn:4831543296}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:4}, log_config_meta:{version:3, proposal_id:4, prev:{version:1, config:{config_version:{proposal_id:4, config_seq:9}, log_sync_memberlist:2{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}{server:"192.168.1.63:2882", timestamp:1, flag:0}, log_sync_replica_num:2, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:4, config_seq:10}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:4, prev_lsn:{lsn:64074526}, prev_mode_pid:4}, log_snapshot_meta:{version:2, base_lsn:{lsn:4831543296}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:0, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:4, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:86016}, readable_log_tail:{lsn:86016}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:5502590976}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1/1/meta", dir_fd:257, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67018752, block_header_serialize_buf_:0x7f7adde62a38, flashback_version:0}, log_storage:{log_tail:{lsn:5463543556}, readable_log_tail:{lsn:5463543556}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:5435486208}, min_scn:{val:1742171570508885001, v:0}, curr_block_id:81, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1/1/log", dir_fd:265, min_block_id:0, max_block_id:82, curr_writable_block_id:81}, logical_block_size_:67104768, curr_block_writable_size_:39047420, block_header_serialize_buf_:0x7f7adde64738, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1741318427753108, this:0x7f7adde60f70}, [Reconfirm]={palf_id:1, self:"192.168.1.62:2882", state:"FINISHED", new_proposal_id:4, prepare_log_ack_list:1"192.168.1.63:2882";, curr_paxos_follower_list:1{server:"192.168.1.63:2882", timestamp:1, flag:0}, majority_cnt:2, majority_max_log_server:"192.168.1.62:2882", majority_max_accept_pid:3, majority_max_lsn:{lsn:64058503}, saved_end_lsn:{lsn:64058503}, last_submit_prepare_req_time_us:1741318427151587, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f7adde609b0}) [2025-03-17 09:57:01.697640] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2751) [117795][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=157] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}, last_ack_time_us:1742176621687096, last_flushed_end_lsn:{lsn:5463543556}}]) [2025-03-17 09:57:01.697652] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4325) [117795][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=11] current log_task status(palf_id=1, self="192.168.1.62:2882", log_id=10438970, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1741318290961698, flag:0}, last_ack_time_us:1742176621687096, last_flushed_end_lsn:{lsn:5463543556}}]) [2025-03-17 09:57:01.712982] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=9] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:01.714081] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=44] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.215425038337707520e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=607, Throttle Trigger(MB)=364, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.298262830575065152e+02) [2025-03-17 09:57:01.714933] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=24][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.714974] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=40][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:01.714994] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:01.715002] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6de0, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63409-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176621714500, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:394, callback_enqueue_used:4, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.715049] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=38][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6de0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63409-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176621714500, enqueue_used:2, dequeue_used:19, submit_used:2, return_used:394, callback_enqueue_used:4, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176621715041}}, part_id:-1}) [2025-03-17 09:57:01.715334] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=30][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:01.715373] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=38][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:01.715393] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=19][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f797409715a, src_buf_size=1611) [2025-03-17 09:57:01.715405] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7578, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63409-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176621715073, enqueue_used:2, dequeue_used:15, submit_used:1, return_used:212, callback_enqueue_used:1, callback_dequeue_used:29, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:01.715486] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=65][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7578, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63409-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176621715073, enqueue_used:2, dequeue_used:15, submit_used:1, return_used:212, callback_enqueue_used:1, callback_dequeue_used:29, callback_finish_used:-1, end_used:1742176621715472}}, part_id:-1}) [2025-03-17 09:57:01.715669] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222489147, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf370d20}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:01.715742] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=55] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634377} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:01.715777] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=29] end stmt(ret=0, tx_id=9634377, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf370d20, tx_id:{txid:9634377}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222489147, assoc_session_id:3222489147, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176621713255, active_ts:1742176621713255, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176651713248, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465828528260, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621710073}], exec_info_reap_ts:1742176621710072, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621710069, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222489147, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf370d20}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:01.715946] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634377}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176651713248, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621713255}{self_ls_id:{id:1}, session_id:3222489147, associated_session_id:3222489147, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634377}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465828528260, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621714383, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634377} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.716027] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=67] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634377}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176651713248, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176621713255}{self_ls_id:{id:1}, session_id:3222489147, associated_session_id:3222489147, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634377}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409465828528260, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176621714383, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634377} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:01.716090] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=56] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634377}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176621715941, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:01.716111] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=18] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf370d20, tx_id:{txid:9634377}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222489147, assoc_session_id:3222489147, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176621713255, active_ts:1742176621713255, commit_ts:-1, finish_ts:1742176621715444, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176651713248, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409465828528260, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176621710073}], exec_info_reap_ts:1742176621710072, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176621710069, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:01.716222] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63409-0-0] [lt=1][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:01.731657] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=24294, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:57:01.731693] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859713799}], scn_array=[cnt:1, {val:1742176621696217000, v:0}], curr_size=588, accum_size=1176, time_cost=34780) [2025-03-17 09:57:01.731747] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=7995) [2025-03-17 09:57:01.731919] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1742176621533386000, v:0}, new_max_version={val:1742176621533386000, v:0}, last_min_version={val:1742176621234020000, v:0}, last_max_version={val:1742176621234020000, v:0}, total_time_us=36196, delta_us=198531) [2025-03-17 09:57:01.731953] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1742176621533386000, v:0}, min_version={val:1742176621533386000, v:0}, max_version={val:1742176621533386000, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:57:01.748799] WDIAG [ARCHIVE] do_thread_task_ (ob_archive_sender.cpp:257) [3539431][T1003_ArcSender][T1003][YB420A0B093E-00062FB7F8D54125-0-0] [lt=22][errcode=-4018] try free send task failed(ret=-4018) [2025-03-17 09:57:01.759579] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859714387}], scn_array=[cnt:1, {val:1742176621733640000, v:0}], curr_size=588, accum_size=1764, time_cost=25856) [2025-03-17 09:57:01.761090] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=33][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463543556}], scn_array=[cnt:1, {val:1742176621747411000, v:0}], curr_size=587, accum_size=12670, time_cost=12715) [2025-03-17 09:57:01.778760] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=23] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:01.781694] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872705938}], scn_array=[cnt:1, {val:1742176621658961001, v:0}], curr_size=122, accum_size=244, time_cost=20868) [2025-03-17 09:57:01.781872] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=9] [APPLY STAT CB APPEND COST TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=38, this=0x7f79ae39c280, extra_info="") [2025-03-17 09:57:01.781901] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=28] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=17085, this=0x7f79ae39c4b0, extra_info="") [2025-03-17 09:57:01.781914] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=14] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=0, this=0x7f79ae39c6e0, extra_info="") [2025-03-17 09:57:01.781923] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=9] [APPLY STAT CB EXECUTE TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=-508, this=0x7f79ae39c910, extra_info="") [2025-03-17 09:57:01.781928] INFO [LIB] stat (utility.h:1210) [3539224][T1004_ApplySrv0][T1004][YB420A0B093E-00062FB8DBD9BBAF-0-0] [lt=5] [APPLY STAT CB TOTAL TIME](cur_stat_count=18, stat_interval=5000000, avg (count/cost)=16616, this=0x7f79ae39cb40, extra_info="") [2025-03-17 09:57:01.781953] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.789844] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=27][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:352, tid:3539397}]) [2025-03-17 09:57:01.808523] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=59] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40307215564, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:01.847588] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=14] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"192.168.1.62:2882", log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2025-03-17 09:57:01.860992] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=20] QSched: |cfg: root:1,x,0 net_in:1,1203M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:336K/78:17030 disk:336K/78:17030 t1m2:156K/35:18551 t1m2g0:156K/35:18551 t1004m2:92K/22:15562 t1004m2g0:92K/22:15562 t1003m2:88K/21:16030 t1003m2g0:88K/21:16030 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:57:01.867023] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=0) [2025-03-17 09:57:01.879595] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=69] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=16860) [2025-03-17 09:57:01.881709] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162417777}], scn_array=[cnt:1, {val:1742176621658961002, v:0}], curr_size=122, accum_size=732, time_cost=20820) [2025-03-17 09:57:01.881894] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706060}], scn_array=[cnt:1, {val:1742176621658961002, v:0}], curr_size=122, accum_size=366, time_cost=21012) [2025-03-17 09:57:01.882033] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.918917] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463544143}], scn_array=[cnt:1, {val:1742176621747411001, v:0}], curr_size=122, accum_size=12792, time_cost=39312) [2025-03-17 09:57:01.934173] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=26][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:57:01.934201] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:57:01.935231] INFO [SHARE.LOCATION] renew_all_ls_locations_by_rpc (ob_ls_location_service.cpp:801) [117410][TimerWK3][T0][YB420A0B093E-00062FB95067DE5F-0-0] [lt=32] [LS_LOCATION] Get ls leaders by RPC(ret=0, ret="OB_SUCCESS", dests=["192.168.1.62:2882", "192.168.1.63:2882", "192.168.1.64:2882", "192.168.1.65:2882"], leaders=[{key:{tenant_id:1, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:4}}, {key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1741272511}, location:{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1741272511}, location:{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1005, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1006, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1006, ls_id:{id:1001}, cluster_id:1741272511}, location:{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}}, {key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}}, {key:{tenant_id:1006, ls_id:{id:1002}, cluster_id:1741272511}, location:{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}}, {key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1741272511}, location:{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}}, {key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1741272511}, location:{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}}, {key:{tenant_id:1006, ls_id:{id:1003}, cluster_id:1741272511}, location:{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}}]) [2025-03-17 09:57:01.939890] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=14] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312806295225, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:01.939933] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=38][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:57:01.939947] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=13] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7aa40492b0) [2025-03-17 09:57:01.939960] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=13] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:0, threads_idle_time:1312806295225, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:01.939946] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859714975}], scn_array=[cnt:1, {val:1742176621896747000, v:0}], curr_size=588, accum_size=2352, time_cost=42834) [2025-03-17 09:57:01.940024] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=7874, avg_group_log_size=605) [2025-03-17 09:57:01.940108] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=11] [APPLY STAT CB APPEND COST TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=37, this=0x7f798c66c280, extra_info="") [2025-03-17 09:57:01.940131] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=22] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=28619, this=0x7f798c66c4b0, extra_info="") [2025-03-17 09:57:01.940139] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=8] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=0, this=0x7f798c66c6e0, extra_info="") [2025-03-17 09:57:01.940161] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=21] [APPLY STAT CB EXECUTE TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=-501, this=0x7f798c66c910, extra_info="") [2025-03-17 09:57:01.940167] INFO [LIB] stat (utility.h:1210) [3539407][T1003_ApplySrv0][T1003][Y0-0000000000000000-0-0] [lt=6] [APPLY STAT CB TOTAL TIME](cur_stat_count=56, stat_interval=5000000, avg (count/cost)=28155, this=0x7f798c66cb40, extra_info="") [2025-03-17 09:57:01.941179] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [117737][T1_Occam][T1][YB420A0B093E-00062FB7EFFFBFD6-0-0] [lt=33] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:01.948473] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND COST TIME](cur_stat_count=24, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:57:01.960912] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:57:01.960959] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:57:01.960973] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:57:01.975720] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [3539266][T1004_DBMSSched][T1004][YB420A0B093E-00062FB7FC7B6B05-0-0] [lt=6] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:57:01.982096] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:01.982765] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859715563}], scn_array=[cnt:1, {val:1742176621933229000, v:0}], curr_size=588, accum_size=2940, time_cost=42735) [2025-03-17 09:57:01.982977] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=38] [PALF STAT WRITE LOG COST TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=19139, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:57:01.982997] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463544265}], scn_array=[cnt:1, {val:1742176621948119000, v:0}], curr_size=586, accum_size=13378, time_cost=34497) [2025-03-17 09:57:01.983022] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=13378) [2025-03-17 09:57:01.983044] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=19, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:57:01.983063] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=13378) [2025-03-17 09:57:01.983094] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=19] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=8, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:57:01.983108] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=13] [PALF STAT LOG LIFE TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=23098, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:57:01.983120] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT FLUSH CB COST TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=39, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:57:01.985019] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15859, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:57:01.985056] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706182}], scn_array=[cnt:1, {val:1742176621658961003, v:0}], curr_size=122, accum_size=488, time_cost=24040) [2025-03-17 09:57:01.985067] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162417899}], scn_array=[cnt:1, {val:1742176621658961003, v:0}], curr_size=122, accum_size=854, time_cost=24069) [2025-03-17 09:57:01.985088] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=28, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:57:01.985130] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=8, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:57:01.985169] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=38] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=15979, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:57:01.985185] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:57:01.985207] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=6, avg_log_submit_to_flush_time=14718, avg_log_submit_to_slide_time=14730) [2025-03-17 09:57:01.985229] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:02.008887] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=33] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40303021260, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:02.012262] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=22] sql audit evict mem start(request_manager_->get_tenant_id()=1004, evict_low_mem_level=278313880, evict_high_mem_level=371085173, size_used=196608, mem_used=372274176) [2025-03-17 09:57:02.012295] INFO [SERVER] release_record (ob_dl_queue.cpp:338) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=30] release ref is not 0 or release_wait_times < 3(rq_release_idx=233, ret=0, release_wait_times_=2, need_clean=false) [2025-03-17 09:57:02.012304] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:206) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=8] release old cannot free more memory [2025-03-17 09:57:02.012310] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=5] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=1, elapse_time=49, size_used=196608, mem_used=372274176) [2025-03-17 09:57:02.012339] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=26] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=233) [2025-03-17 09:57:02.021930] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117913][T1_L0_G0][T1][YB420A0B093F-00062FB7B25CA587-0-0] [lt=12][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:02.052700] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1089) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=29] replay status is not follower(min_unreplayed_lsn={lsn:0}, base_lsn={lsn:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595957515623, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79ae32fa70, log_storage_:{log_tail:{lsn:872706304}, readable_log_tail:{lsn:872706304}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:872361984}, min_scn:{val:1742176383843265001, v:0}, curr_block_id:13, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1/log", dir_fd:275, min_block_id:0, max_block_id:14, curr_writable_block_id:13}, logical_block_size_:67104768, curr_block_writable_size_:66760448, block_header_serialize_buf_:0x7f79ae330738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.052801] INFO [CLOG] operator() (ob_log_replay_service.cpp:1679) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=100] get_replay_process success(id={id:1}, submitted_log_size=0, unsubmitted_log_size=0, replayed_log_size=0, unreplayed_log_size=0) [2025-03-17 09:57:02.052811] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1089) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=8] replay status is not follower(min_unreplayed_lsn={lsn:0}, base_lsn={lsn:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595979082691, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741595966225063000, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79c4aa1a70, log_storage_:{log_tail:{lsn:4162418021}, readable_log_tail:{lsn:4162418021}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4160495616}, min_scn:{val:1742176364319009003, v:0}, curr_block_id:62, palf_id:1001}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1001/log", dir_fd:274, min_block_id:0, max_block_id:63, curr_writable_block_id:62}, logical_block_size_:67104768, curr_block_writable_size_:65182363, block_header_serialize_buf_:0x7f79c4aa2738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1001, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.052844] INFO [CLOG] operator() (ob_log_replay_service.cpp:1679) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=33] get_replay_process success(id={id:1001}, submitted_log_size=0, unsubmitted_log_size=0, replayed_log_size=0, unreplayed_log_size=0) [2025-03-17 09:57:02.052863] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:182) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=5] dump tenant replay process(tenant_id=1004, unsubmitted_log_size(MB)=0, unreplayed_log_size(MB)=0, estimate_time(second)=0, submitted_log_size(MB)=0, replayed_log_size(MB)=0, last_submitted_log_size(MB)=0, last_replayed_log_size(MB)=0, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2025-03-17 09:57:02.061021] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:02.063886] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=30] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:02.063927] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=29] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:02.071389] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:728) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=24] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=12100000, replace_num=100000) [2025-03-17 09:57:02.071417] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=26] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=10) [2025-03-17 09:57:02.079702] INFO [LIB] stat (utility.h:1210) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=25, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:57:02.082164] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.082694] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162418021}], scn_array=[cnt:1, {val:1742176621658961004, v:0}], curr_size=122, accum_size=976, time_cost=21654) [2025-03-17 09:57:02.082739] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=11][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706304}], scn_array=[cnt:1, {val:1742176621658961004, v:0}], curr_size=122, accum_size=610, time_cost=21741) [2025-03-17 09:57:02.085010] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539346][T1004_L0_G0][T1004][YB420A0B093E-00062FB8DBD9BBCE-0-0] [lt=23] get number(ret=0, service_type_=0, range=1, base_id=1742176622084124000, start_id=1742176622084124000, end_id=1742176622084124001) [2025-03-17 09:57:02.097278] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=19] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:57:02.118871] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=92] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:02.127313] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:02.127373] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=33] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:6, fail_bw:24531, [delay/us]:prepare:1, schedule:28, submit:1, rt:767, total:798, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:02.127393] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4742, iops:19, bw:89950, [delay/us]:prepare:5, schedule:29, submit:1096, rt:21244, total:22375) [2025-03-17 09:57:02.127401] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=4] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:02.127447] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=42] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:6, avg_bw:24531, [delay/us]: prepare:1, schedule:28, submit:1, device:767, total:798) [2025-03-17 09:57:02.127465] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4743, avg_iops:19, avg_bw:89950, [delay/us]: prepare:5, schedule:29, submit:1096, device:21244, total:22375) [2025-03-17 09:57:02.127473] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=3] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:57:02.127502] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32709, [delay/us]:prepare:1, schedule:23, submit:1, rt:571, total:598, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:02.127515] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4778, iops:12, bw:57241, [delay/us]:prepare:5, schedule:24, submit:160, rt:29070, total:29261) [2025-03-17 09:57:02.127523] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:02.127541] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32710, [delay/us]: prepare:1, schedule:23, submit:1, device:571, total:598) [2025-03-17 09:57:02.127555] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4779, avg_iops:12, avg_bw:57243, [delay/us]: prepare:5, schedule:24, submit:160, device:29070, total:29261) [2025-03-17 09:57:02.127565] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:57:02.127582] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8177, [delay/us]:prepare:6, schedule:13, submit:1, rt:12793, total:12813) [2025-03-17 09:57:02.127602] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4300, iops:20, bw:85865, [delay/us]:prepare:5, schedule:22, submit:1979, rt:11817, total:13824) [2025-03-17 09:57:02.127610] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS TENANT](tenant_id=1004, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33434400, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:02.127628] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8177, [delay/us]: prepare:6, schedule:13, submit:1, device:12793, total:12813) [2025-03-17 09:57:02.127642] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4301, avg_iops:20, avg_bw:85868, [delay/us]: prepare:5, schedule:22, submit:1979, device:11817, total:13824) [2025-03-17 09:57:02.127654] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:57:02.127693] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=16] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4088, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:4388, total:4395) [2025-03-17 09:57:02.127724] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:02.127756] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4088, [delay/us]: prepare:0, schedule:0, submit:600000000, device:4388, total:4395) [2025-03-17 09:57:02.127775] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:57:02.127787] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:02.127795] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127801] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127808] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127814] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127820] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127826] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127833] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127840] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:02.127847] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127853] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127859] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127866] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:02.127874] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS SERVER] net_in=48498kB/s, net_out=328kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:57:02.130141] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=12478464) [2025-03-17 09:57:02.130210] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=49] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:02.131140] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162418143}], scn_array=[cnt:1, {val:1742176622084124000, v:0}], curr_size=1381, accum_size=2357, time_cost=45998) [2025-03-17 09:57:02.140685] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859716151}], scn_array=[cnt:1, {val:1742176622096914000, v:0}], curr_size=588, accum_size=3528, time_cost=43395) [2025-03-17 09:57:02.153761] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:449) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=8] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:02.153781] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=19] try reload schema success(schema_version=1741661297536408, refresh_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_ret=0) [2025-03-17 09:57:02.153790] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:263) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A2-0-0] [lt=11] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:02.153815] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:288) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A2-0-0] [lt=23] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:02.160187] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=27] gts statistics(tenant_id=1, gts_rpc_cnt=10, get_gts_cache_cnt=1001, get_gts_with_stc_cnt=75, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=69, try_wait_gts_elapse_cnt=0) [2025-03-17 09:57:02.160228] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=39] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:57:02.160253] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=13] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1003, need_refresh=false, gts_local_cache={srr:[mts=1742176622160240], gts:1742176622159882000, latest_srr:[mts=1742176622160240]}) [2025-03-17 09:57:02.160605] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=14] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_refresh=false, gts_local_cache={srr:[mts=1742176622160585], gts:1742176622159882000, latest_srr:[mts=1742176622160585]}) [2025-03-17 09:57:02.166540] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859716739}], scn_array=[cnt:1, {val:1742176622133279000, v:0}], curr_size=588, accum_size=4116, time_cost=25734) [2025-03-17 09:57:02.166586] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463544973}], scn_array=[cnt:1, {val:1742176622148158000, v:0}], curr_size=587, accum_size=709, time_cost=18113) [2025-03-17 09:57:02.174790] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=26] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:57:02.177032] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:559) [117778][T1_HBService][T1][YB420A0B093E-00062FB7F27BCA1B-0-0] [lt=22] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2025-03-17 09:57:02.182058] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=27] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:57:02.182254] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.190077] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706426}], scn_array=[cnt:1, {val:1742176622159882000, v:0}], curr_size=122, accum_size=732, time_cost=28988) [2025-03-17 09:57:02.195494] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC7-0-0] [lt=28] [RS_MGR] force set rs list(master_rs="192.168.1.62:2882") [2025-03-17 09:57:02.195524] INFO [SERVER] handle_heartbeat (ob_service.cpp:3753) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC7-0-0] [lt=26] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"192.168.1.62:2882", server_id:4, rs_addr:"192.168.1.62:2882", rs_server_status:1, epoch_id:4}, hb_response={zone:"zone1", server:"192.168.1.62:2882", sql_port:2881, build_version:"4.3.5.0_100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5(Dec 31 2024 17:35:01)", start_service_time:1741316885024955, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=32) [2025-03-17 09:57:02.209221] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40303021260, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:02.214006] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=37] PNIO [ratelimit] time: 1742176622214004, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007602, add_bytes: 0 [2025-03-17 09:57:02.216879] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:671) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=31] Cache wash clean map node details(ret=0, clean_node_count=1, clean_time=0, clean_start_pos=6000000, clean_num=200000) [2025-03-17 09:57:02.246636] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:02.246661] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:02.247521] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFBA-0-0] [lt=6][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:02.247555] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=19][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:02.247566] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:57:02.253796] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=24] ====== check clog disk timer task ====== [2025-03-17 09:57:02.253819] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=19] get_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:57:02.271521] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=22] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=9) [2025-03-17 09:57:02.280257] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706548}], scn_array=[cnt:1, {val:1742176622159882001, v:0}], curr_size=122, accum_size=854, time_cost=19097) [2025-03-17 09:57:02.280717] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162420104}], scn_array=[cnt:1, {val:1742176622159882000, v:0}], curr_size=122, accum_size=3059, time_cost=19525) [2025-03-17 09:57:02.282319] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.293107] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463545560}], scn_array=[cnt:1, {val:1742176622159882000, v:0}], curr_size=122, accum_size=831, time_cost=13170) [2025-03-17 09:57:02.293171] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107787, offset=57288309}, end_cursor:ObLogCursor{file_id=5, log_id=1107788, offset=57288615}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17734050, len:2443, log_data_len:306, seq:1107787, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:02.293248] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=0] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8778, ref_cnt_:1, raw_buf_:0x7f7ac2552170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78626-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622289237, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:3978, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:02.293306] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=56][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.293327] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=19][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.293331] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=26] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:57:02.293346] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:02.293359] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=17] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176622159882000, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176622159882000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:02.293353] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8778, ref_cnt_:1, raw_buf_:0x7f7ac2552170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78626-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622289237, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:3978, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.293393] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8778, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78626-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622289237, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:3978, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176622293389}}, part_id:-1}) [2025-03-17 09:57:02.293399] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622289237, enqueue_used:1, dequeue_used:17, submit_used:1, return_used:3978, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:165, end_used:-22}}) [2025-03-17 09:57:02.293462] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=61][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:02.293476] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:02.293782] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.293805] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=22][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:02.293811] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7ac6b6d510, src_buf_size=295) [2025-03-17 09:57:02.293816] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8b20, ref_cnt_:1, raw_buf_:0x7f7ac6b6c150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78626-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622293491, enqueue_used:1, dequeue_used:18, submit_used:1, return_used:260, callback_enqueue_used:2, callback_dequeue_used:7, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.293852] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=26][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8b20, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78626-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622293491, enqueue_used:1, dequeue_used:18, submit_used:1, return_used:260, callback_enqueue_used:2, callback_dequeue_used:7, callback_finish_used:-1, end_used:1742176622293844}}, part_id:-1}) [2025-03-17 09:57:02.293864] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=10][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176622293491, enqueue_used:1, dequeue_used:18, submit_used:1, return_used:260, callback_enqueue_used:2, callback_dequeue_used:7, callback_finish_used:93, end_used:-29}}) [2025-03-17 09:57:02.293892] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=28][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:02.293901] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=7][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a41072080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.293928] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=27][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a41072080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.293954] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=23][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:02.293968] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=13][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cb697d0}) [2025-03-17 09:57:02.293987] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=18][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cb6a578, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:0, rowkey_end_idx:0, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cb6a578, micro_handle:NULL}) [2025-03-17 09:57:02.294021] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=33][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:57:02.294028] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=7][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a60348a58, local_datums:0x7f7a60348a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:57:02.294175] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=146][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7a60348a58, local_datums:0x7f7a60348a58}, datums:[]}) [2025-03-17 09:57:02.294185] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=10][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176622159882000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:57:02.294191] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7a60347ac0) [2025-03-17 09:57:02.294196] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=4][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:57:02.294202] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=4][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1},{"BIGINT":1};{"BIGINT":1},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176652187989, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7a603473d0, valid:true, source:2, core:{version:{val:1742176622159882000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176622287909, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7a603439a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7a603476a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7a60343848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7a60347ac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:57:02.294452] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=249][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:02.294457] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:57:02.294462] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=4][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:57:02.294466] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=3][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:57:02.294473] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:57:02.294480] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:02.294487] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a6033fb50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:02.294500] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=12][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:57:02.294512] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=11][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:57:02.294519] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:57:02.294532] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=12][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:57:02.294538] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7a6033fb50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:57:02.294552] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=13][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:02.294561] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=8][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:02.294570] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=8][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:57:02.294580] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=9][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1"}) [2025-03-17 09:57:02.294592] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=11][errcode=-4016] execute failed(ret=-4016, tenant_id=1, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741661297536408) [2025-03-17 09:57:02.294605] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=11][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:02.294630] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=10] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade9f2520, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222521951, assoc_session_id:3222521951, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176622288802, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176622287909, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7a2b3c6f58, id:3222521951, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade9f2520}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.294685] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=52][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:02.294691] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=5][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:02.294694] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=3][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:02.294704] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=4][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:02.294713] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=9][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1"}, aret=-4016, ret=-4016) [2025-03-17 09:57:02.294722] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:1806) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=7][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1) [2025-03-17 09:57:02.294730] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=7][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1) [2025-03-17 09:57:02.294737] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1746) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=6][errcode=-4016] execute_read failed(ret=-4016, cluster_id=1741272511, tenant_id=1) [2025-03-17 09:57:02.294745] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:138) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=7][errcode=-4016] query failed(ret=-4016, conn=0x7f7b317c6060, start=1742176622289106, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1) [2025-03-17 09:57:02.294753] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:67) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=8][errcode=-4016] read failed(ret=-4016) [2025-03-17 09:57:02.294758] WDIAG [SHARE] exec_read (ob_ls_i_life_manager.h:213) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=4][errcode=-4016] failed to read(ret=-4016, ret="OB_ERR_UNEXPECTED", exec_tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1) [2025-03-17 09:57:02.294814] WDIAG [SHARE] get_ls_primary_zone_info (ob_ls_status_operator.cpp:1062) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78626-0-0] [lt=8][errcode=-4016] failed to read ls recovery(ret=-4016, ret="OB_ERR_UNEXPECTED", tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1) [2025-03-17 09:57:02.295163] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=39][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:02.295613] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=18] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=7761) [2025-03-17 09:57:02.297529] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=26] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:57:02.298251] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=27] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:57:02.298321] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=24] inner load succ(latest_freeze_info={frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176617637226000, v:0}, frozen_statuses_.count():6, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741314277005166000, v:0}, schema_version:1741314275575456, data_version:17180067072}, {frozen_version:{val:1741370403684882000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741456800355942000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741543202134877000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}]}) [2025-03-17 09:57:02.298344] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:719) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=21] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1742176617637226000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741314277005166000, v:0}, schema_version:1741314275575456, data_version:17180067072}, {frozen_version:{val:1741370403684882000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741456800355942000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741543202134877000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1, freeze_info:{latest_snapshot_gc_scn:{val:1742176617637226000, v:0}, frozen_statuses_.count():6, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741314277005166000, v:0}, schema_version:1741314275575456, data_version:17180067072}, {frozen_version:{val:1741370403684882000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741456800355942000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741543202134877000, v:0}, schema_version:1741317197577304, data_version:17180067072}, {frozen_version:{val:1741629603950087000, v:0}, schema_version:1741595966851560, data_version:17180067072}]}}) [2025-03-17 09:57:02.305912] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=25] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:57:02.313338] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=32] PNIO [ratelimit] time: 1742176622313336, bytes: 2535113208572, bw: 46.902061 MB/s, add_ts: 1003573, add_bytes: 49356097 [2025-03-17 09:57:02.325119] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7C-0-0] [lt=9] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:57:02.325172] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7C-0-0] [lt=51] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:57:02.331195] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=48][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859717327}], scn_array=[cnt:1, {val:1742176622295227000, v:0}], curr_size=1132, accum_size=5248, time_cost=35566) [2025-03-17 09:57:02.338451] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] [COMPACTION DAG STATUS] (start_time=1742176612336405, end_time=1742176622338450, dag_status={}) [2025-03-17 09:57:02.338480] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=27] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338491] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338497] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338504] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338510] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338516] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338523] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338529] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338534] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:02.338541] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338550] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338560] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE/MEDIUM_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338570] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338580] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338589] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338599] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338609] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338618] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338628] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"VERIFY_CKM", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338638] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:4, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338648] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"UPDATE_SKIP_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338657] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"REFRESH_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338667] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338677] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338687] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338696] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338705] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_DEL_LOB_META", dag_module_str:"DROP_VEC_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338715] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338725] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338734] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338744] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATE_WARMUP", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338753] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"WAIT_DATA_READY", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338763] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338772] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338782] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338791] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338801] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338811] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338820] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338830] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338840] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338849] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338859] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338868] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_CHECKE_CONVERT", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338878] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338887] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338897] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338910] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338922] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338935] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338948] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338961] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338973] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338983] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.338993] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339002] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339012] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339022] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"INITIAL_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339031] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"START_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339040] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_TABLET_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339050] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"FINISH_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339063] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339075] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339088] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339101] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339120] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339131] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339141] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=18] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339169] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339179] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339189] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339198] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339206] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339215] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339225] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339235] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BUILD_TABLET_INFO", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339261] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339273] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:8, sys_task_type:24, dag_type_str:"VECTOR_INDEX_DAG", dag_module_str:"VECTOR_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:02.339298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3635) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=13] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2025-03-17 09:57:02.339310] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=11] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2025-03-17 09:57:02.339320] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2025-03-17 09:57:02.339330] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=10] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2025-03-17 09:57:02.339339] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=9] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2025-03-17 09:57:02.339347] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2025-03-17 09:57:02.339354] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2025-03-17 09:57:02.339361] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2025-03-17 09:57:02.339368] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2025-03-17 09:57:02.339375] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2025-03-17 09:57:02.339383] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4342) [3539385][T1003_DagSchedu][T1003][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(dag_cnt=0, total_worker_cnt_=2, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2025-03-17 09:57:02.347227] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=9] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:02.348035] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=26] gc stale ls task succ [2025-03-17 09:57:02.348069] INFO [STORAGE.TRANS] alloc (ob_trans_factory.cpp:141) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=32] ObTransCtx statistics(active_coord_ctx_count=0, active_part_ctx_count=3, total_release_part_ctx_count=152) [2025-03-17 09:57:02.348992] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=42][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.349024] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=30][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.349044] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=19][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:02.349055] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9348, ref_cnt_:1, raw_buf_:0x7f7a22814150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622348463, enqueue_used:2, dequeue_used:40, submit_used:1, return_used:455, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:02.349104] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=41][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9348, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622348463, enqueue_used:2, dequeue_used:40, submit_used:1, return_used:455, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:-1, end_used:1742176622349099}}, part_id:-1}) [2025-03-17 09:57:02.349109] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=15][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622348463, enqueue_used:2, dequeue_used:40, submit_used:1, return_used:455, callback_enqueue_used:3, callback_dequeue_used:25, callback_finish_used:142, end_used:-32}}) [2025-03-17 09:57:02.349156] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=45][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:02.349166] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:02.358339] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859718459}], scn_array=[cnt:1, {val:1742176622297353000, v:0}], curr_size=588, accum_size=5836, time_cost=27029) [2025-03-17 09:57:02.358430] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=34] Slow write(duration=26918, write_len=4096) [2025-03-17 09:57:02.358455] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=24] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107788, offset=57288615}, end_cursor:ObLogCursor{file_id=5, log_id=1107789, offset=57288912}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17708050, len:2137, log_data_len:297, seq:1107788, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:02.358460] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=21] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:57:02.358474] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=4] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=2461, avg_log_gen_to_submit_time=2476, avg_log_submit_to_flush_time=18143, avg_log_submit_to_slide_time=18164) [2025-03-17 09:57:02.358545] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=5233, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:02.358553] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.358583] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=29][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:02.358592] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a0e5bd15a, src_buf_size=1611) [2025-03-17 09:57:02.358599] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c84a8, ref_cnt_:1, raw_buf_:0x7f7a0e5bc150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622349174, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:9327, callback_enqueue_used:3, callback_dequeue_used:27, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:02.358634] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=17] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:57:02.358633] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=26][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c84a8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340A-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622349174, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:9327, callback_enqueue_used:3, callback_dequeue_used:27, callback_finish_used:-1, end_used:1742176622358628}}, part_id:-1}) [2025-03-17 09:57:02.358660] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731D-0-0] [lt=14] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176622134336000, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176622134336000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:02.358642] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=6][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aa4274080, time_log:{begin_ts:1742176622349174, enqueue_used:1, dequeue_used:16, submit_used:1, return_used:9327, callback_enqueue_used:3, callback_dequeue_used:27, callback_finish_used:103, end_used:-24}}) [2025-03-17 09:57:02.358684] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=41][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:02.358694] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.358712] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=17][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7aa4274080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.358728] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=12][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:02.358740] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=11][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:02.358759] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=18][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:02.358803] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=43][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:57:02.358809] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=5][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:02.358815] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=5][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:02.358823] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:02.358831] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652248306, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:02.358906] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=74][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652248306, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: FB03B61A80300600, int: 1742176622347259}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}}) [2025-03-17 09:57:02.358950] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=42][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: FB03B61A80300600, int: 1742176622347259}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:57:02.358972] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=21][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:02.358978] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=5][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:57:02.358983] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}, ret=-4016) [2025-03-17 09:57:02.359013] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=6][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7a22944e90, local_datums:0x7f7a22944ae0}, datums:[col_id=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: FB03B61A80300600, int: 1742176622347259}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944961, hex: FB03B61A80300600, int: 1742176622347259}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944969, hex: FB03B61A80300600, int: 1742176622347259}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944971, hex: 0700000000000000, int: 7}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944979, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7a22944981, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7a22944987, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 54, flag: 0, null: 0, ptr: 0x7f7a22944998, hex: 44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029, cstr: DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7a229449ce, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7a229449d7, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7a229449df, hex: 00D826AD68300600, int: 1742076000000000}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7a229449e7, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7a229449f8, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176652248306, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176652248306, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176622333382000, v:0}, tx_id:{txid:9634387}, scn:{branch:0, seq:2160}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634387}, tx_desc:{this:0x7f7acf3cfec0, tx_id:{txid:9634387}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513509, assoc_session_id:3222513509, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176622347156, active_ts:1742176622347156, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652347150, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466462544955, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176622348224}], exec_info_reap_ts:1742176622348223, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622346065, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:2162}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:02.359093] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=78][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:57:02.359114] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=18][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: FB03B61A80300600, int: 1742176622347259}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 13, flag: 0, null: 0, ptr: 0x7f7a2294494c, hex: 53554E4441595F57494E444F57, cstr: SUNDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944959, hex: FB03B61A80300600, int: 1742176622347259}], store_rowkey:{"VARCHAR":"SUNDAY_WINDOW", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.347259"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:2415161696, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652248306, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:57:02.359177] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=62][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:57:02.359182] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:57:02.359191] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:02.359197] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=5][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:57:02.359209] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:02.359213] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:57:02.359217] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:02.359222] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:57:02.359226] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=4][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176652348306, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56388972, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf3cfec0, tx_id:{txid:9634387}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513509, assoc_session_id:3222513509, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176622347156, active_ts:1742176622347156, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652347150, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466462544955, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2162}, last_touch_ts:1742176622348224}], exec_info_reap_ts:1742176622348223, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622346065, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634387}, scn:{branch:0, seq:2160}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176652248306, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176652248306, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176652248306, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:561, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:57:02.360175] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=948][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:57:02.360186] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=10][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:02.360201] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=14][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:02.360209] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:57:02.360216] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:57:02.360225] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:57:02.360236] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=10][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:57:02.360251] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=14][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:02.360259] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:57:02.360267] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:57:02.360274] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:02.360282] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:02.360290] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=7][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}) [2025-03-17 09:57:02.360301] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=11][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:57:02.360316] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=12][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:02.360336] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=13] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222513509, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3cfec0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:02.360383] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:12795}, to_seq_no={branch:0, seq:2160}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634387} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:02.360430] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=35] end stmt(ret=0, tx_id=9634387, plain_select=false, stmt_type=2, savepoint={branch:0, seq:2160}, tx_desc={this:0x7f7acf3cfec0, tx_id:{txid:9634387}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513509, assoc_session_id:3222513509, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176622347156, active_ts:1742176622347156, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652347150, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466462544955, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2160}, last_touch_ts:1742176622348224}], exec_info_reap_ts:1742176622348223, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622346065, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222513509, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3cfec0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.360482] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=50][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:02.360490] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:02.360495] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=5][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:02.360508] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:02.360519] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=10][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')"}, aret=-4016, ret=-4016) [2025-03-17 09:57:02.360529] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=9][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW')) [2025-03-17 09:57:02.360547] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=9][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:57:02.360556] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=8][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW'), is_user_sql=false) [2025-03-17 09:57:02.360570] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=13][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176622347259), usec_to_time(1742176622347259), 7, usec_to_time(1742176622347259), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E4741544845525F44415441424153455F53544154535F4A4F425F50524F4328373230303030303030303029', 'SCHEDULED', usec_to_time(1742076000000000), 'SUNDAY_WINDOW'), conn=0x7f7990f8ed90) [2025-03-17 09:57:02.360584] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=13][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:57:02.360632] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=6][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634387}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176652347150, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622347156}{self_ls_id:{id:1}, session_id:3222513509, associated_session_id:3222513509, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634387}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466462544955, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622348223, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634387} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.360711] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=70] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634387}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176652347150, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622347156}{self_ls_id:{id:1}, session_id:3222513509, associated_session_id:3222513509, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634387}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466462544955, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622348223, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634387} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.360764] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=48] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634387}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176622360628, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:02.360781] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=16] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3cfec0, tx_id:{txid:9634387}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513509, assoc_session_id:3222513509, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176622347156, active_ts:1742176622347156, commit_ts:-1, finish_ts:1742176622359918, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652347150, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466462544955, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:2160}, last_touch_ts:1742176622348224}], exec_info_reap_ts:1742176622348223, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622346065, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:02.360863] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=32][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:57:02.360872] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=9][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:57:02.360884] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340A-0-0] [lt=11][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:02.365355] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=13666, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:57:02.365382] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=3181) [2025-03-17 09:57:02.365399] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=23, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:57:02.365410] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=3181) [2025-03-17 09:57:02.365451] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=7, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:57:02.365464] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=13756, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:57:02.365481] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=33, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:57:02.376954] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706670}], scn_array=[cnt:1, {val:1742176622159882002, v:0}], curr_size=122, accum_size=976, time_cost=15749) [2025-03-17 09:57:02.382392] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.395945] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=50][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859719635}], scn_array=[cnt:1, {val:1742176622359918000, v:0}], curr_size=748, accum_size=7172, time_cost=30805) [2025-03-17 09:57:02.415134] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.415769] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.415774] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=21] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.415896] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.415919] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.415919] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=25] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.416231] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.417245] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=35] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40296729804, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:02.429612] INFO [CLOG] get_replay_process (ob_replay_status.cpp:1089) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=24] replay status is not follower(min_unreplayed_lsn={lsn:64058503}, base_lsn={lsn:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:1, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741318427097633, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:64058503}, next_to_submit_scn_:{val:1741318426647365001, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741314277197622000, v:0}, iterator_:{iterator_impl:{buf_:0x7f7aef805000, next_round_pread_size:3678208, curr_read_pos:1011, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1011, log_storage_:{start_lsn:{lsn:64057492}, end_lsn:{lsn:64058503}, read_buf:{buf_len_:2932736, buf_:0x7f7aef805000}, block_size:67104768, log_storage_:0x7f7adde63a70, log_storage_:{log_tail:{lsn:5463546269}, readable_log_tail:{lsn:5463546269}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:5435486208}, min_scn:{val:1742171570508885001, v:0}, curr_block_id:81, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1/1/log", dir_fd:265, min_block_id:0, max_block_id:82, curr_writable_block_id:81}, logical_block_size_:67104768, curr_block_writable_size_:39044707, block_header_serialize_buf_:0x7f7adde64738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1741318426647365000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:2, log_size:923, scn_:{val:1741318426647365000, v:0}, data_checksum:7529717, flag:64521}}, init_mode_version:0, accumulate_checksum:983882438, curr_entry_is_padding:0, padding_entry_size:955, padding_entry_scn:{val:1741318426647365000, v:0}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:8104, miss_cnt:0, cache_read_size:64058503, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.429684] INFO [CLOG] operator() (ob_log_replay_service.cpp:1679) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=72] get_replay_process success(id={id:1}, submitted_log_size=64058503, unsubmitted_log_size=0, replayed_log_size=64058503, unreplayed_log_size=0) [2025-03-17 09:57:02.429707] INFO [CLOG] runTimerTask (ob_log_replay_service.cpp:182) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=8] dump tenant replay process(tenant_id=1, unsubmitted_log_size(MB)=0, unreplayed_log_size(MB)=0, estimate_time(second)=0, submitted_log_size(MB)=61, replayed_log_size(MB)=61, last_submitted_log_size(MB)=61, last_replayed_log_size(MB)=61, round_cost_time(second)=10, pending_replay_log_size(MB)=0) [2025-03-17 09:57:02.433017] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=25] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:02.450598] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:02.452666] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=19][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.452725] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=58][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.452750] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=24][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:02.452762] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6b58, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340B-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452169, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:457, callback_enqueue_used:4, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:02.452816] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=46][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6b58, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340B-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452169, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:457, callback_enqueue_used:4, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176622452811}}, part_id:-1}) [2025-03-17 09:57:02.452821] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=46][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452169, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:457, callback_enqueue_used:4, callback_dequeue_used:9, callback_finish_used:185, end_used:-37}}) [2025-03-17 09:57:02.452873] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=51][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:02.452890] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=15][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:02.453131] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.453243] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=79][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:02.453255] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=11][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aa427515a, src_buf_size=1611) [2025-03-17 09:57:02.453263] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8c40, ref_cnt_:1, raw_buf_:0x7f7aa4274150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340B-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452900, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:210, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:02.453349] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=74][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8c40, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340B-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452900, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:210, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:1742176622453339}}, part_id:-1}) [2025-03-17 09:57:02.453363] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=7][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622452900, enqueue_used:1, dequeue_used:11, submit_used:1, return_used:210, callback_enqueue_used:1, callback_dequeue_used:6, callback_finish_used:-1, end_used:1742176622453339}}) [2025-03-17 09:57:02.453385] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=21][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:02.453394] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=7][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a22814080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.453429] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=35][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a22814080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:02.453456] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=24][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:02.453470] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=13][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:02.453491] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=21][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:02.453537] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=46][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:57:02.453543] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=5][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:02.453549] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=5][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:02.453557] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=4][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:02.453563] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=7][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652351999, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:02.453618] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=55][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652351999, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: D597B71A80300600, int: 1742176622450645}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}}) [2025-03-17 09:57:02.453664] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=44][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: D597B71A80300600, int: 1742176622450645}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:57:02.453689] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=24][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:02.453698] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=8][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:57:02.453707] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=7][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}, ret=-4016) [2025-03-17 09:57:02.453740] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=9][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7aa426ae10, local_datums:0x7f7aa426aa60}, datums:[col_id=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: D597B71A80300600, int: 1742176622450645}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8fd, hex: D597B71A80300600, int: 1742176622450645}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a905, hex: D597B71A80300600, int: 1742176622450645}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a90d, hex: 0800000000000000, int: 8}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a915, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7aa426a91d, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a923, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 28, flag: 0, null: 0, ptr: 0x7f7aa426a934, hex: 44424D535F53544154532E50555247455F5354415453284E554C4C29, cstr: DBMS_STATS.PURGE_STATS(NULL)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a950, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a959, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a961, hex: 1227F4050D300600, int: 1741682352858898}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a969, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a97a, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176652351999, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176652351999, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176622359918000, v:0}, tx_id:{txid:9634391}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634391}, tx_desc:{this:0x7f7acf214ca0, tx_id:{txid:9634391}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222522109, assoc_session_id:3222522109, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176622450789, active_ts:1742176622450789, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652450783, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466566304354, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176622449726}], exec_info_reap_ts:1742176622449725, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622449722, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:02.453827] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=85][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:57:02.453836] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=7][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: D597B71A80300600, int: 1742176622450645}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: D597B71A80300600, int: 1742176622450645}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:02.450645"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176652351999, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:57:02.453887] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=50][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:57:02.453892] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=5][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:57:02.453900] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=4][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:02.453928] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222522109, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf214ca0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:02.453957] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=16] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:3203}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634391} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:02.453987] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=26] end stmt(ret=0, tx_id=9634391, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf214ca0, tx_id:{txid:9634391}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222522109, assoc_session_id:3222522109, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176622450789, active_ts:1742176622450789, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652450783, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466566304354, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622449726}], exec_info_reap_ts:1742176622449725, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622449722, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222522109, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf214ca0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.454092] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634391}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176652450783, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622450789}{self_ls_id:{id:1}, session_id:3222522109, associated_session_id:3222522109, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634391}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466566304354, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622451867, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634391} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.454175] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=72] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634391}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176652450783, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622450789}{self_ls_id:{id:1}, session_id:3222522109, associated_session_id:3222522109, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634391}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466566304354, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622451867, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634391} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.454215] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=35] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634391}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176622454089, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:02.454229] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=13] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf214ca0, tx_id:{txid:9634391}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222522109, assoc_session_id:3222522109, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176622450789, active_ts:1742176622450789, commit_ts:-1, finish_ts:1742176622453984, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652450783, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466566304354, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622449726}], exec_info_reap_ts:1742176622449725, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622449722, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:02.454302] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340B-0-0] [lt=1][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:02.466125] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=33] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}]) [2025-03-17 09:57:02.471633] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=8) [2025-03-17 09:57:02.474479] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162420348}], scn_array=[cnt:1, {val:1742176622159882002, v:0}], curr_size=122, accum_size=122, time_cost=13173) [2025-03-17 09:57:02.480029] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=32] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3666, avg_group_log_size=333) [2025-03-17 09:57:02.480568] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872706792}], scn_array=[cnt:1, {val:1742176622159882003, v:0}], curr_size=122, accum_size=1098, time_cost=19289) [2025-03-17 09:57:02.481912] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=24] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:02.481919] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=23] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:02.481944] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=20] temporary file module use no blocks [2025-03-17 09:57:02.481948] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=31] temporary file module use no blocks [2025-03-17 09:57:02.481952] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:02.481958] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=9] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:02.481963] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=9] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:02.481971] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=12] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:02.481968] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=16] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:02.481990] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=19] temporary file module use no blocks [2025-03-17 09:57:02.481974] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=11] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863117373685, flush_io_finished_ret_:0, flush_io_finished_round_:859380, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:02.481982] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=10] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863117373685, flush_io_finished_ret_:0, flush_io_finished_round_:580430, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:02.481998] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=8] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:02.482010] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=11] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:02.482023] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863117373730, flush_io_finished_ret_:0, flush_io_finished_round_:580431, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:1, normal_idle_loop_cnt_:1, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:02.482471] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=40][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.512444] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=21] sql audit evict mem start(request_manager_->get_tenant_id()=1004, evict_low_mem_level=278313880, evict_high_mem_level=371085173, size_used=196608, mem_used=372274176) [2025-03-17 09:57:02.512478] INFO [SERVER] release_record (ob_dl_queue.cpp:334) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=28] release ref is 0(rq_release_idx=233, need_clean=false, ret=0) [2025-03-17 09:57:02.512489] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:206) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=10] release old cannot free more memory [2025-03-17 09:57:02.512500] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=10] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=648806, evict_batch_count=1, elapse_time=56, size_used=131072, mem_used=372274176) [2025-03-17 09:57:02.512538] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=10] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:02.528678] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463546269}], scn_array=[cnt:1, {val:1742176622348223001, v:0}], curr_size=122, accum_size=1540, time_cost=48633) [2025-03-17 09:57:02.530121] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=38] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:57:02.530187] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=49] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=11883) [2025-03-17 09:57:02.541216] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859720383}], scn_array=[cnt:1, {val:1742176622496750000, v:0}], curr_size=588, accum_size=7760, time_cost=43686) [2025-03-17 09:57:02.541266] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=49] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=7760) [2025-03-17 09:57:02.541296] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=24, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:57:02.541370] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=27] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=8, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:57:02.541396] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT LOG LIFE TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=38968, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:57:02.541404] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=12, avg_log_gen_to_freeze_time=6653, avg_log_gen_to_submit_time=6666, avg_log_submit_to_flush_time=32272, avg_log_submit_to_slide_time=32302) [2025-03-17 09:57:02.541440] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=26] [PALF STAT FLUSH CB COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=57, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:57:02.541433] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=20][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.541454] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=32] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=7760, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:02.541477] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=42][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.541499] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=22][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:02.541509] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8d60, ref_cnt_:1, raw_buf_:0x7f7a6033e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0076-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176622535781, enqueue_used:2, dequeue_used:38, submit_used:1, return_used:5564, callback_enqueue_used:3, callback_dequeue_used:30, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.541553] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=34][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8d60, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0076-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176622535781, enqueue_used:2, dequeue_used:38, submit_used:1, return_used:5564, callback_enqueue_used:3, callback_dequeue_used:30, callback_finish_used:-1, end_used:1742176622541546}}, part_id:-1}) [2025-03-17 09:57:02.541604] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=32][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:02.542035] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.542059] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=25][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:02.542065] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a6033f510, src_buf_size=295) [2025-03-17 09:57:02.542071] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c81d8, ref_cnt_:1, raw_buf_:0x7f7a6033e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0076-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176622541635, enqueue_used:1, dequeue_used:13, submit_used:1, return_used:358, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.542095] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=19][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c81d8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0076-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7aaacbe080, time_log:{begin_ts:1742176622541635, enqueue_used:1, dequeue_used:13, submit_used:1, return_used:358, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:-1, end_used:1742176622542091}}, part_id:-1}) [2025-03-17 09:57:02.542221] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0076-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8a0680, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222522289, assoc_session_id:3222522289, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176622535086, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176622535085, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222522289, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace8a0680}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.543070] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=30] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:57:02.558196] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=17][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.558245] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=47][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.558269] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=23][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:02.558278] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9150, ref_cnt_:1, raw_buf_:0x7f7ac8630170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1D-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176622553120, enqueue_used:1, dequeue_used:49, submit_used:1, return_used:4994, callback_enqueue_used:2, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.558338] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=44][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9150, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1D-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176622553120, enqueue_used:1, dequeue_used:49, submit_used:1, return_used:4994, callback_enqueue_used:2, callback_dequeue_used:25, callback_finish_used:-1, end_used:1742176622558326}}, part_id:-1}) [2025-03-17 09:57:02.558347] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=1][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:02.559733] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:02.559774] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=40][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:02.559783] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7ac4e05510, src_buf_size=295) [2025-03-17 09:57:02.559791] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8928, ref_cnt_:1, raw_buf_:0x7f7ac4e04150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1D-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176622558390, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:1302, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:02.559841] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=41][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8928, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1D-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176622558390, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:1302, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176622559835}}, part_id:-1}) [2025-03-17 09:57:02.559906] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1D-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6b7af00, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222515154, assoc_session_id:3222515154, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176622552115, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176622552114, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222515154, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6b7af00}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:02.564061] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=21] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=1, size_used=65536, mem_used=126864384) [2025-03-17 09:57:02.564254] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:02.569983] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=33] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:57:02.570039] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:57:02.570085] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:02.582539] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.591746] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [3539166][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=29] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}, {1001,0,False,{False,},False,False,True}]) [2025-03-17 09:57:02.599713] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=7] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1) [2025-03-17 09:57:02.599745] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=29] start lib cache evict(tenant_id=1, mem_hold=283115520, mem_limit=322122545, cache_obj_num=776, cache_node_num=773) [2025-03-17 09:57:02.599756] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=10] end lib cache evict(tenant_id=1, cache_evict_num=0, mem_hold=283115520, mem_limit=322122545, cache_obj_num=776, cache_node_num=773) [2025-03-17 09:57:02.599768] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=11] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:02.603519] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=7] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:02.602660] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=8] Dumping All Cache Objs(alloc_obj_list.count()=776, alloc_obj_list=[{obj_id:51, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196394}, {obj_id:72, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180596}, {obj_id:74, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188388}, {obj_id:139, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160477}, {obj_id:329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:327911}, {obj_id:381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189321}, {obj_id:399, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178578}, {obj_id:410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179268}, {obj_id:447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194846}, {obj_id:495, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:557, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196117}, {obj_id:50913, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177580}, {obj_id:53754, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176496}, {obj_id:54672, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233845}, {obj_id:54692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235254}, {obj_id:6335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185047}, {obj_id:6380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190234}, {obj_id:57035, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:276727}, {obj_id:57055, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184180}, {obj_id:57801, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194188}, {obj_id:58087, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180124}, {obj_id:58344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199622}, {obj_id:59240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187102}, {obj_id:59241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:273946}, {obj_id:59242, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197398}, {obj_id:59249, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:280549}, {obj_id:59255, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196978}, {obj_id:59257, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:288566}, {obj_id:59259, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:334006}, {obj_id:59260, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175776}, {obj_id:59261, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:317353}, {obj_id:59304, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195857}, {obj_id:59875, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177585}, {obj_id:59876, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177622}, {obj_id:59879, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178040}, {obj_id:59881, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235344}, {obj_id:59882, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187939}, {obj_id:59884, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183047}, {obj_id:59890, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178857}, {obj_id:59895, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194979}, {obj_id:59897, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235364}, {obj_id:59900, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199510}, {obj_id:59905, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209961}, {obj_id:59907, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195635}, {obj_id:59908, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59909, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59910, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59912, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59917, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200616}, {obj_id:59918, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195188}, {obj_id:59919, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179136}, {obj_id:59920, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177462}, {obj_id:59921, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188401}, {obj_id:59922, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193417}, {obj_id:59923, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59924, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178328}, {obj_id:59925, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59926, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182265}, {obj_id:59927, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192859}, {obj_id:59928, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59929, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248320}, {obj_id:59930, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59931, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185648}, {obj_id:59932, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213471}, {obj_id:59935, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194647}, {obj_id:59936, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177264}, {obj_id:59937, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185959}, {obj_id:59938, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248440}, {obj_id:59939, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226192}, {obj_id:59940, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176800}, {obj_id:59941, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:174944}, {obj_id:59942, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177893}, {obj_id:59943, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194037}, {obj_id:59944, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181347}, {obj_id:59945, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185263}, {obj_id:59946, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185681}, {obj_id:59947, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196901}, {obj_id:59948, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203466}, {obj_id:59949, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193243}, {obj_id:59950, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189090}, {obj_id:59951, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190242}, {obj_id:59952, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194564}, {obj_id:59958, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59959, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185063}, {obj_id:59960, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59962, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:59971, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:59972, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:252561}, {obj_id:59978, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:59979, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176224}, {obj_id:59981, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186299}, {obj_id:59982, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179554}, {obj_id:59984, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200201}, {obj_id:59985, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213559}, {obj_id:59986, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:257040}, {obj_id:59988, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:59989, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179720}, {obj_id:59998, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207279}, {obj_id:60000, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278740}, {obj_id:60001, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183087}, {obj_id:60002, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:267931}, {obj_id:60003, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:251136}, {obj_id:60005, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:324244}, {obj_id:60007, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179606}, {obj_id:60008, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:3, added_to_lc:true, mem_used:492243}, {obj_id:60009, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60010, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:60011, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:60012, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:60015, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179886}, {obj_id:60016, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178336}, {obj_id:60017, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181705}, {obj_id:60018, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:60022, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178376}, {obj_id:60023, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:60024, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178376}, {obj_id:60025, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:60026, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:60027, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:275212}, {obj_id:60028, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249175}, {obj_id:60029, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185679}, {obj_id:60052, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:60053, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177800}, {obj_id:60054, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204741}, {obj_id:60056, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215059}, {obj_id:60061, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180166}, {obj_id:60064, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195605}, {obj_id:60067, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204726}, {obj_id:60069, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240092}, {obj_id:60078, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:249595}, {obj_id:60080, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193936}, {obj_id:60332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203439}, {obj_id:60333, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:60424, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200032}, {obj_id:60442, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197358}, {obj_id:60660, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180250}, {obj_id:60662, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180358}, {obj_id:60673, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192624}, {obj_id:60674, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190259}, {obj_id:60686, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192062}, {obj_id:60688, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192617}, {obj_id:60690, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196328}, {obj_id:60691, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205892}, {obj_id:60692, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185222}, {obj_id:60743, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204639}, {obj_id:60980, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233446}, {obj_id:61026, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200592}, {obj_id:61082, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185236}, {obj_id:61083, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222880}, {obj_id:61084, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180498}, {obj_id:61089, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181704}, {obj_id:61091, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206237}, {obj_id:61092, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184745}, {obj_id:61093, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183368}, {obj_id:61123, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197825}, {obj_id:61124, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181456}, {obj_id:61125, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189172}, {obj_id:61126, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61127, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61128, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61129, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61130, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61131, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61132, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61134, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224140}, {obj_id:61135, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183490}, {obj_id:61176, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196918}, {obj_id:61181, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202039}, {obj_id:61182, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:61183, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180267}, {obj_id:61184, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176208}, {obj_id:61185, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193061}, {obj_id:61192, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199220}, {obj_id:61193, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182816}, {obj_id:61194, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187579}, {obj_id:61195, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225220}, {obj_id:61197, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180400}, {obj_id:61198, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182234}, {obj_id:61199, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176192}, {obj_id:61200, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178808}, {obj_id:61201, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178464}, {obj_id:61202, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194564}, {obj_id:61206, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180187}, {obj_id:61207, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184050}, {obj_id:61208, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179059}, {obj_id:61209, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179107}, {obj_id:61210, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183376}, {obj_id:61212, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184262}, {obj_id:61213, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177856}, {obj_id:61214, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178608}, {obj_id:61216, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182348}, {obj_id:61217, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181360}, {obj_id:61218, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177688}, {obj_id:61219, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175744}, {obj_id:61221, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198173}, {obj_id:61233, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178568}, {obj_id:61234, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178568}, {obj_id:61238, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176496}, {obj_id:61239, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195916}, {obj_id:61240, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191698}, {obj_id:61241, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61242, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203003}, {obj_id:61243, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179032}, {obj_id:61244, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179032}, {obj_id:61245, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61246, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202624}, {obj_id:61247, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203148}, {obj_id:61248, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238146}, {obj_id:61249, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240110}, {obj_id:61250, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177088}, {obj_id:61251, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176763}, {obj_id:61252, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177200}, {obj_id:61253, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:61254, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:61255, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180574}, {obj_id:61256, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185347}, {obj_id:61257, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187858}, {obj_id:61258, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204326}, {obj_id:61259, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223580}, {obj_id:61260, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197140}, {obj_id:61261, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185593}, {obj_id:61263, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178739}, {obj_id:61264, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207006}, {obj_id:61265, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:237389}, {obj_id:61266, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189942}, {obj_id:61267, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:238589}, {obj_id:61268, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214079}, {obj_id:61269, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:61270, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:61271, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209930}, {obj_id:61272, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187946}, {obj_id:61273, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236433}, {obj_id:61274, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199161}, {obj_id:61275, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177864}, {obj_id:61276, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178589}, {obj_id:61277, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201976}, {obj_id:61278, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:273840}, {obj_id:61279, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186702}, {obj_id:61280, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184321}, {obj_id:61281, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182076}, {obj_id:61282, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182688}, {obj_id:61283, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204022}, {obj_id:61284, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176448}, {obj_id:61285, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217616}, {obj_id:61286, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224439}, {obj_id:61287, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192995}, {obj_id:61288, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240962}, {obj_id:61289, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182421}, {obj_id:61290, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223189}, {obj_id:61291, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198587}, {obj_id:61292, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203764}, {obj_id:61293, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211287}, {obj_id:61294, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182312}, {obj_id:61295, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:61296, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:61297, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199853}, {obj_id:61298, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:175776}, {obj_id:61299, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211964}, {obj_id:61300, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201388}, {obj_id:61301, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:61302, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211967}, {obj_id:61303, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:240913}, {obj_id:61304, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206969}, {obj_id:61305, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195404}, {obj_id:61306, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61307, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208488}, {obj_id:61308, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205131}, {obj_id:61309, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196241}, {obj_id:61310, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61311, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199179}, {obj_id:61312, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207224}, {obj_id:61313, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192816}, {obj_id:61314, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184947}, {obj_id:61315, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219225}, {obj_id:61316, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:494025}, {obj_id:61317, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180312}, {obj_id:61318, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:486704}, {obj_id:61319, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:485788}, {obj_id:61320, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1080567}, {obj_id:61321, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177104}, {obj_id:61322, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176224}, {obj_id:61323, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182929}, {obj_id:61324, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:61326, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:61328, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182195}, {obj_id:61329, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182404}, {obj_id:61331, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:61332, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:61333, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180908}, {obj_id:61334, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181123}, {obj_id:61335, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:61336, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:61337, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:61338, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:61339, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:61340, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:61341, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:290643}, {obj_id:61342, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230505}, {obj_id:61343, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:300745}, {obj_id:61344, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:243566}, {obj_id:61345, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210085}, {obj_id:61346, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221070}, {obj_id:61347, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208917}, {obj_id:61348, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61349, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61350, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61351, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61352, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61353, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61354, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61355, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61356, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61357, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61358, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61359, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61360, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61361, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61362, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61363, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207763}, {obj_id:61364, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186121}, {obj_id:61365, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:381025}, {obj_id:61366, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217225}, {obj_id:61367, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202386}, {obj_id:61368, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61369, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61370, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216347}, {obj_id:61371, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204192}, {obj_id:61372, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176528}, {obj_id:61373, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200801}, {obj_id:61374, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61375, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203691}, {obj_id:61376, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61377, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61378, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61379, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61380, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61381, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202155}, {obj_id:61382, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61383, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61384, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61385, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61386, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205810}, {obj_id:61387, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258426}, {obj_id:61388, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236555}, {obj_id:61389, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178496}, {obj_id:61390, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178248}, {obj_id:61391, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:215450}, {obj_id:61392, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206477}, {obj_id:61393, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201310}, {obj_id:61394, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210377}, {obj_id:61395, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61396, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61397, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61398, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61399, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61400, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180200}, {obj_id:61401, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61402, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61403, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213211}, {obj_id:61404, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210945}, {obj_id:61405, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:61406, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212648}, {obj_id:61407, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:61408, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:254425}, {obj_id:61409, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235203}, {obj_id:61410, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61411, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209121}, {obj_id:61412, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61413, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211805}, {obj_id:61414, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61415, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61416, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:205861}, {obj_id:61417, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61418, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61419, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61420, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61421, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211504}, {obj_id:61422, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61423, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61424, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61425, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61426, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61427, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:219069}, {obj_id:61428, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61429, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61430, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217282}, {obj_id:61431, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194233}, {obj_id:61432, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198403}, {obj_id:61433, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196768}, {obj_id:61434, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61435, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61436, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61437, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:212753}, {obj_id:61438, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213756}, {obj_id:61439, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61440, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61441, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61442, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61443, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61444, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61445, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214979}, {obj_id:61446, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185353}, {obj_id:61447, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213332}, {obj_id:61448, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61449, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61450, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61451, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61452, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61453, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216791}, {obj_id:61454, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61455, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61456, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61457, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61458, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61459, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61460, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61461, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61462, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61463, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61464, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61465, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61466, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218192}, {obj_id:61467, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61468, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61469, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180016}, {obj_id:61470, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61471, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61472, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61473, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61474, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61475, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61476, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61477, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61478, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61479, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61480, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61481, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:61482, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:256432}, {obj_id:61483, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61484, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61485, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199405}, {obj_id:61486, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176952}, {obj_id:61487, tenant_id:1, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, [2025-03-17 09:57:02.604034] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=1372] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:02.605192] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=31] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9b28, ref_cnt_:1, raw_buf_:0x7f79740b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340C-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a22814080, time_log:{begin_ts:1742176622604679, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:477, callback_enqueue_used:3, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:02.605251] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=57][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:02.605279] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=27][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.605714] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222516918, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf21ee80}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:02.605766] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=38] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1059}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634394} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:02.605803] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=34] end stmt(ret=0, tx_id=9634394, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf21ee80, tx_id:{txid:9634394}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516918, assoc_session_id:3222516918, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176622603245, active_ts:1742176622603245, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652603239, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466718840057, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622603248}], exec_info_reap_ts:1742176622603247, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622603244, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222516918, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf21ee80}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.605910] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634394}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176652603239, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622604302}{self_ls_id:{id:1}, session_id:3222516918, associated_session_id:3222516918, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634394}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466718840057, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622604302, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634394} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.605973] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=52] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634394}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176652603239, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622604302}{self_ls_id:{id:1}, session_id:3222516918, associated_session_id:3222516918, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634394}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466718840057, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622604302, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634394} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.606011] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=34] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634394}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176622605906, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:02.606027] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf21ee80, tx_id:{txid:9634394}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516918, assoc_session_id:3222516918, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176622603245, active_ts:1742176622603245, commit_ts:-1, finish_ts:1742176622605366, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176652603239, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466718840057, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622603248}], exec_info_reap_ts:1742176622603247, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622603244, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:02.606096] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340C-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:02.607931] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=46, len=268, pos=238) [2025-03-17 09:57:02.608514] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=27][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=24, pos=15) [2025-03-17 09:57:02.609068] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=10, len=125, pos=123) [2025-03-17 09:57:02.609616] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=17][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=394, pos=298) [2025-03-17 09:57:02.610367] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=12][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=15, len=187, pos=169) [2025-03-17 09:57:02.611020] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F571E-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=0, len=4, pos=4) [2025-03-17 09:57:02.611958] INFO [PALF] runTimerTask (block_gc_timer_task.cpp:101) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=10] BlockGCTimerTask success(ret=0, cost_time_us=11, palf_env_impl_={IPalfEnvImpl:{IPalfEnvImpl:"Dummy"}, self:"192.168.1.62:2882", log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1", disk_options_wrapper:{disk_opts_for_stopping_writing:{log_disk_size(MB):8192, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):8192, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):666, sequence:2}, log_alloc_mgr_:{flying_log_task:0, flying_meta_task:0}}) [2025-03-17 09:57:02.617591] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=32] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40292535500, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:02.630305] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=45] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=0, size_used=0, mem_used=22877184) [2025-03-17 09:57:02.630365] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=45] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:02.635206] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.635298] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=27] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.635335] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=21] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.635337] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636020] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=52] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636162] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=26] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636228] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636382] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636460] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=18] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.636455] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:02.641699] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=20] gc stale ls task succ [2025-03-17 09:57:02.655665] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=67] ====== traversal_flush timer task ====== [2025-03-17 09:57:02.655794] INFO [MDS] flush (mds_table_mgr.cpp:195) [117781][T1_TimerWK0][T1][YB420A0B093E-00062FB91A28D8C7-0-0] [lt=23] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742067170807625001, v:0}}, }, max_consequent_callbacked_scn={val:1742176622548869000, v:0}, *this={this:0x7f7ac1de7150, is_inited:true, freezing_scn:{val:1741629603822325000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636588679000, latest_log_ts:{val:1742176619945902001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.655878] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [117781][T1_TimerWK0][T1][YB420A0B093E-00062FB91A28D8C7-0-0] [lt=88] succeed to traversal_flush(ret=0, ls_cnt=1) [2025-03-17 09:57:02.660968] INFO [STORAGE.TRANS] handle_request (ob_timestamp_service.cpp:175) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=16] handle gts request statistics(total_rt=940, total_cnt=229, avg_rt=4.086956521739130821e+00, avg_cnt=2.289999999999999858e+01) [2025-03-17 09:57:02.661397] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:02.661457] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=56] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:02.661488] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=11, avg_log_batch_cnt=1, total_group_log_size=3059, avg_group_log_size=278) [2025-03-17 09:57:02.661502] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=3059) [2025-03-17 09:57:02.671750] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=32] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=7) [2025-03-17 09:57:02.673749] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=21] ====== checkpoint timer task ====== [2025-03-17 09:57:02.673788] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=23] get_max_applied_scn(scn={val:1742176622548869000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:4, palf_committed_end_lsn_:{lsn:5463546977}, palf_committed_end_scn_:{val:1742176622548869000, v:0}, last_check_scn_:{val:1742176622548869000, v:0}, max_applied_cb_scn_:{val:1742176622548869000, v:0}}) [2025-03-17 09:57:02.673819] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1042) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=27] get_min_unreplayed_log_info(lsn={lsn:64058503}, scn={val:1741318426647365001, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741318427097633, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:64058503}, next_to_submit_scn_:{val:1741318426647365001, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741314277197622000, v:0}, iterator_:{iterator_impl:{buf_:0x7f7aef805000, next_round_pread_size:3678208, curr_read_pos:1011, curr_read_buf_start_pos:0, curr_read_buf_end_pos:1011, log_storage_:{start_lsn:{lsn:64057492}, end_lsn:{lsn:64058503}, read_buf:{buf_len_:2932736, buf_:0x7f7aef805000}, block_size:67104768, log_storage_:0x7f7adde63a70, log_storage_:{log_tail:{lsn:5463546977}, readable_log_tail:{lsn:5463546977}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:5435486208}, min_scn:{val:1742171570508885001, v:0}, curr_block_id:81, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1/1/log", dir_fd:265, min_block_id:0, max_block_id:82, curr_writable_block_id:81}, logical_block_size_:67104768, curr_block_writable_size_:39043999, block_header_serialize_buf_:0x7f7adde64738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:1741318426647365000, v:0}, curr_entry:{LogEntryHeader:{magic:19528, version:2, log_size:923, scn_:{val:1741318426647365000, v:0}, data_checksum:7529717, flag:64521}}, init_mode_version:0, accumulate_checksum:983882438, curr_entry_is_padding:0, padding_entry_size:955, padding_entry_scn:{val:1741318426647365000, v:0}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:8104, miss_cnt:0, cache_read_size:64058503, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.674267] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=27] gc stale ls task succ [2025-03-17 09:57:02.674543] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=75] succ to get rec scn(*this={this:0x7f7ac9e04030, ls_id:{id:1}, tenant_id:1, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1741318426647365001, v:0}, max_applied_start_working_ts_:{val:1741318426647365001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741318291584325, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:1078}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.674581] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=39] tx ctx memtable get rec scn(this={ObITable:{this:0x7f7ac579a080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1741629604055327, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f7ac579a080, snapshot_version:{val:1741629604055327, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.674666] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=18] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742067170807625001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742067170807625001, v:0}}, }, *this={this:0x7f7ac1de7150, is_inited:true, freezing_scn:{val:1741629603822325000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636588679000, latest_log_ts:{val:1742176619945902001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.674731] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=63] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="DATA_CHECKPOINT_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7f7ac1df4ad0}, min_rec_scn={val:1742067167143792000, v:0}, ls_id_={id:1}) [2025-03-17 09:57:02.674744] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=10] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:0, v:0}) [2025-03-17 09:57:02.674772] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=26] get rec log scn(service_type_=0, rec_log_ts={val:1742176619945902001, v:0}) [2025-03-17 09:57:02.674777] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=5] get rec log scn(service_type_=1, rec_log_ts={val:1742139624749258001, v:0}) [2025-03-17 09:57:02.674782] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=4] get rec log scn(service_type_=2, rec_log_ts={val:1742138690841156001, v:0}) [2025-03-17 09:57:02.674792] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=6] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:02.674821] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:02.674839] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742067167143792000, v:0}, checkpoint_scn_in_ls_meta={val:1742067167143792000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:57:02.674855] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=9][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4831543296}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:0, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4831543296}, lsn={lsn:4831543296}) [2025-03-17 09:57:02.674914] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=58] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4831543296}, ls->get_ls_id()={id:1}) [2025-03-17 09:57:02.674929] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=6] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2025-03-17 09:57:02.675830] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=19] ====== checkpoint timer task ====== [2025-03-17 09:57:02.675866] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=22] get_max_applied_scn(scn={val:1742176622159882004, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:872707036}, palf_committed_end_scn_:{val:1742176622159882004, v:0}, last_check_scn_:{val:1742176622661033000, v:0}, max_applied_cb_scn_:{val:1742176622159882004, v:0}}) [2025-03-17 09:57:02.675882] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=13] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595957515623, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79ae32fa70, log_storage_:{log_tail:{lsn:872707036}, readable_log_tail:{lsn:872707036}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:872361984}, min_scn:{val:1742176383843265001, v:0}, curr_block_id:13, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1/log", dir_fd:275, min_block_id:0, max_block_id:14, curr_writable_block_id:13}, logical_block_size_:67104768, curr_block_writable_size_:66759716, block_header_serialize_buf_:0x7f79ae330738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.676637] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=70] succ to get rec scn(*this={this:0x7f79fc004030, ls_id:{id:1}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:2, v:0}, max_applied_start_working_ts_:{val:2, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595954890076, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:713}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.676669] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=32] tx ctx memtable get rec scn(this={ObITable:{this:0x7f79ae3a8080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f79ae3a8080, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.676814] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=18] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742148082680599002, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1742148082680599002, v:0}}, }, *this={this:0x7f79ae2db150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176620157204000, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176637787617000, latest_log_ts:{val:1742176621158150001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.676884] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=67] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7f79ae2da1b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7f79ae2e5010, memtable_head:19, memtable_tail:20, t3m:0x7f79f86ea030, tables:[null, null, null, 0x7f79fac1a7b0, null, null, null, null, null, null, null, null, null, null, null, null]}, is_freezing:false, ls_id:{id:1}, mini_merge_recycle_commit_versions_ts:1742148004181715, tx_data_table:0x7f79ae2e6d10, ls_tablet_svr:0x7f79ae2da190}, min_rec_scn={val:1742148008655744000, v:0}, ls_id_={id:1}) [2025-03-17 09:57:02.676907] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=20] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1, v:0}) [2025-03-17 09:57:02.676920] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=12] get rec log scn(service_type_=0, rec_log_ts={val:1742176621158150001, v:0}) [2025-03-17 09:57:02.676929] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=9] get rec log scn(service_type_=1, rec_log_ts={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.676940] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=7] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:02.676953] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:02.676963] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=9] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742148008655744000, v:0}, checkpoint_scn_in_ls_meta={val:1742148008655744000, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:57:02.676979] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:805257216}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:805257216}, lsn={lsn:805257216}) [2025-03-17 09:57:02.677008] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=28] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:805257216}, ls->get_ls_id()={id:1}) [2025-03-17 09:57:02.677029] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=11] get_max_applied_scn(scn={val:1742176622159882003, v:0}, this={ls_id_:{id:1001}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:4162420592}, palf_committed_end_scn_:{val:1742176622159882003, v:0}, last_check_scn_:{val:1742176622661033000, v:0}, max_applied_cb_scn_:{val:1742176622159882003, v:0}}) [2025-03-17 09:57:02.677052] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=21] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1741595966225063000, v:0}, this={ls_id_:{id:1001}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595979082691, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1741595966225063000, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79c4aa1a70, log_storage_:{log_tail:{lsn:4162420592}, readable_log_tail:{lsn:4162420592}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4160495616}, min_scn:{val:1742176364319009003, v:0}, curr_block_id:62, palf_id:1001}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1001/log", dir_fd:274, min_block_id:0, max_block_id:63, curr_writable_block_id:62}, logical_block_size_:67104768, curr_block_writable_size_:65179792, block_header_serialize_buf_:0x7f79c4aa2738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1001, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:02.677786] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=71] succ to get rec scn(*this={this:0x7f79fc0b6030, ls_id:{id:1001}, tenant_id:1004, tx_ls_state_mgr:{ls_id_:{id:1001}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:1741595966225063001, v:0}, max_applied_start_working_ts_:{val:1741595966225063001, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595976091006, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:246}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.677817] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=31] tx ctx memtable get rec scn(this={ObITable:{this:0x7f79ae3a9350, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:2, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f79ae3a9350, snapshot_version:{val:4611686018427387903, v:0}, ls_id:{id:1001}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:02.677872] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=16] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742148095278530005, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:200048}, rec_scn:{val:1742148095278530005, v:0}}, }, *this={this:0x7f79e6d15150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148081676232000, v:0}, clog_base_lsn:{lsn:4093390848}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176620157204000, v:0}, tablet_change_checkpoint_scn:{val:1742008506458625001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.677931] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=58] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="TX_DATA_MEMTABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={ObIMemtableMgr:{Memtables:this:0x7f79e6d141b0, ref_cnt:1, is_inited:true, tablet_id:{id:49402}, freezer:0x7f79e6d1f010, memtable_head:12, memtable_tail:13, t3m:0x7f79f86ea030, tables:[null, null, null, null, null, null, null, null, null, null, null, null, 0x7f79fac18080, null, null, null]}, is_freezing:false, ls_id:{id:1001}, mini_merge_recycle_commit_versions_ts:1742148004203272, tx_data_table:0x7f79e6d20d10, ls_tablet_svr:0x7f79e6d14190}, min_rec_scn={val:1742148081676232000, v:0}, ls_id_={id:1001}) [2025-03-17 09:57:02.677950] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1001}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1741595966225063000, v:0}) [2025-03-17 09:57:02.677960] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=8] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2025-03-17 09:57:02.677973] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=11] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1001}) [2025-03-17 09:57:02.677985] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=11] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742148081676232000, v:0}, checkpoint_scn_in_ls_meta={val:1742148081676232000, v:0}, ls_id={id:1001}, service_type="TRANS_SERVICE") [2025-03-17 09:57:02.677999] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4093390848}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1741595966225063000, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4093390848}, lsn={lsn:4093390848}) [2025-03-17 09:57:02.678015] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=16] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4093390848}, ls->get_ls_id()={id:1001}) [2025-03-17 09:57:02.678024] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=9] succeed to update_clog_checkpoint(ret=0, ls_cnt=2) [2025-03-17 09:57:02.679123] INFO [SQL.EXE] run2 (ob_maintain_dependency_info_task.cpp:228) [117536][MaintainDepInfo][T0][Y0-0000000000000000-0-0] [lt=26] [ASYNC TASK QUEUE](queue_.size()=0, sys_view_consistent_.size()=518) [2025-03-17 09:57:02.679913] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:775) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=30] success to update min reserved snapshot(reserved_snapshot=1742174815071995000, duration=1800, snapshot_gc_ts=1742176615071995000) [2025-03-17 09:57:02.682660] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.683243] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162420592}], scn_array=[cnt:1, {val:1742176622661033000, v:0}], curr_size=122, accum_size=366, time_cost=21744) [2025-03-17 09:57:02.683305] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=9, len=124, pos=91) [2025-03-17 09:57:02.683632] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872707036}], scn_array=[cnt:1, {val:1742176622661033000, v:0}], curr_size=122, accum_size=122, time_cost=22225) [2025-03-17 09:57:02.683851] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=21][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=484, pos=300) [2025-03-17 09:57:02.684307] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=17][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=19, len=219, pos=189) [2025-03-17 09:57:02.684745] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=67, len=312, pos=282) [2025-03-17 09:57:02.685337] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=23][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=77, pos=48) [2025-03-17 09:57:02.685819] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=395, pos=299) [2025-03-17 09:57:02.686299] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5720-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=35, len=248, pos=219) [2025-03-17 09:57:02.690490] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859721559}], scn_array=[cnt:1, {val:1742176622533984001, v:0}], curr_size=112, accum_size=700, time_cost=10493) [2025-03-17 09:57:02.690552] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463546977}], scn_array=[cnt:1, {val:1742176622659975000, v:0}], curr_size=122, accum_size=2248, time_cost=10356) [2025-03-17 09:57:02.690603] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:802) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=27] update reserved snapshot finished(cost_ts=10692, reserved_snapshot=1742174815071995000) [2025-03-17 09:57:02.697383] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=29] current monitor number(seq_=-1) [2025-03-17 09:57:02.714032] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=15] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:02.715194] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=35] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.196225056648254395e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=598, Throttle Trigger(MB)=358, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.383279685020984857e+02) [2025-03-17 09:57:02.716047] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:02.716504] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222513351, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2a4740}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:02.716551] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=30] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634396} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:02.716578] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=24] end stmt(ret=0, tx_id=9634396, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2a4740, tx_id:{txid:9634396}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513351, assoc_session_id:3222513351, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176622714585, active_ts:1742176622714585, commit_ts:-1, finish_ts:-1, timeout_us:29999991, lock_timeout_us:-1, expire_ts:1742176652714576, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466829639506, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622714588}], exec_info_reap_ts:1742176622714587, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622714584, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222513351, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2a4740}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:02.716700] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634396}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176652714576, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622714585}{self_ls_id:{id:1}, session_id:3222513351, associated_session_id:3222513351, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634396}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466829639506, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622714585, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634396} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.716785] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=70] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634396}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176652714576, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176622714585}{self_ls_id:{id:1}, session_id:3222513351, associated_session_id:3222513351, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634396}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409466829639506, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176622714585, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634396} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:02.716836] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=47] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634396}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176622716695, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:02.716852] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2a4740, tx_id:{txid:9634396}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222513351, assoc_session_id:3222513351, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176622714585, active_ts:1742176622714585, commit_ts:-1, finish_ts:1742176622716717, timeout_us:29999991, lock_timeout_us:-1, expire_ts:1742176652714576, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409466829639506, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176622714588}], exec_info_reap_ts:1742176622714587, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176622714584, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:02.716928] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340D-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:02.717753] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859721671}], scn_array=[cnt:1, {val:1742176622697524000, v:0}], curr_size=588, accum_size=1288, time_cost=19616) [2025-03-17 09:57:02.720688] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:817) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=16] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}) [2025-03-17 09:57:02.751932] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT WRITE LOG COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=26627, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:57:02.751951] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=19][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859722259}], scn_array=[cnt:1, {val:1742176622733915000, v:0}], curr_size=588, accum_size=1876, time_cost=16999) [2025-03-17 09:57:02.751995] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8460) [2025-03-17 09:57:02.752141] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=32] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1004, new_min_version={val:1742176622159882002, v:0}, new_max_version={val:1742176622159882002, v:0}, last_min_version={val:1742176622159882001, v:0}, last_max_version={val:1742176622159882001, v:0}, total_time_us=18681, delta_us=592258) [2025-03-17 09:57:02.752177] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=34] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1004, current_version={val:1742176622159882002, v:0}, min_version={val:1742176622159882002, v:0}, max_version={val:1742176622159882002, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:57:02.759393] INFO [PALF] is_allowed_broadcast_ (palf_handle_impl.h:1404) [117745][T1_Occam][T1][Y0-0000000000000000-0-0] [lt=57] keep in silent because of disconnected with RS, do not solicit votes.(is_in_silent=false, palf_id=1, is_allowed_broadcast=true) [2025-03-17 09:57:02.762066] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463547099}], scn_array=[cnt:1, {val:1742176622748755000, v:0}], curr_size=587, accum_size=2835, time_cost=13134) [2025-03-17 09:57:02.778894] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=38] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:02.782688] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.817915] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=34] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40290438348, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:02.819653] INFO [STORAGE] runTimerTask (ob_locality_manager.cpp:708) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=9] runTimer to refresh locality_info(ret=0) [2025-03-17 09:57:02.820740] INFO [SHARE] set_server_locality_array (ob_server_locality_cache.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=19] set_server_locality_array success(server_locality_array=[{inited:true, addr:"192.168.1.62:2882", zone:"zone1", zone_type:0, idc:"default_idc", region:"sys_region", is_active:true, start_service_time:1741316885024955, server_stop_time:0, server_status:1}, {inited:true, addr:"192.168.1.63:2882", zone:"zone1", zone_type:0, idc:"default_idc", region:"sys_region", is_active:true, start_service_time:1741313538300339, server_stop_time:0, server_status:1}, {inited:true, addr:"192.168.1.64:2882", zone:"zone1", zone_type:0, idc:"default_idc", region:"sys_region", is_active:true, start_service_time:1741767271024436, server_stop_time:0, server_status:1}, {inited:true, addr:"192.168.1.65:2882", zone:"zone1", zone_type:0, idc:"default_idc", region:"sys_region", is_active:true, start_service_time:1741767273737049, server_stop_time:0, server_status:1}], has_readonly_zone=false) [2025-03-17 09:57:02.820779] INFO [SHARE] load_region (ob_locality_table_operator.cpp:336) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=34] tenant_schema primary_zone is NULL, or no need calc region priority(tenant_id=1, primary_zone=RANDOM) [2025-03-17 09:57:02.820787] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=7] add_locality_zone success(locality_zone={tenant_id:1, region_priority:18446744073709551615}) [2025-03-17 09:57:02.820799] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}]}) [2025-03-17 09:57:02.820812] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=12] add_locality_zone success(locality_zone={tenant_id:1001, region_priority:0}) [2025-03-17 09:57:02.820819] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}]}) [2025-03-17 09:57:02.820831] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=11] add_locality_zone success(locality_zone={tenant_id:1002, region_priority:0}) [2025-03-17 09:57:02.820836] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=3] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}, {tenant_id:1002, region_priority:0}]}) [2025-03-17 09:57:02.820853] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=17] add_locality_zone success(locality_zone={tenant_id:1003, region_priority:0}) [2025-03-17 09:57:02.820859] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=4] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}, {tenant_id:1002, region_priority:0}, {tenant_id:1003, region_priority:0}]}) [2025-03-17 09:57:02.820870] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=11] add_locality_zone success(locality_zone={tenant_id:1004, region_priority:0}) [2025-03-17 09:57:02.820879] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}, {tenant_id:1002, region_priority:0}, {tenant_id:1003, region_priority:0}, {tenant_id:1004, region_priority:0}]}) [2025-03-17 09:57:02.820890] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=12] add_locality_zone success(locality_zone={tenant_id:1005, region_priority:0}) [2025-03-17 09:57:02.820898] INFO [SHARE] load_region (ob_locality_table_operator.cpp:346) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=7] get_primary_zone_prioriry(primary_zone=zone1, tenant_primary_region_array=[{region:"sys_region", region_priority:0, zone_array:["zone1"]}], locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}, {tenant_id:1002, region_priority:0}, {tenant_id:1003, region_priority:0}, {tenant_id:1004, region_priority:0}, {tenant_id:1005, region_priority:0}]}) [2025-03-17 09:57:02.820910] INFO [SHARE] load_region (ob_locality_table_operator.cpp:363) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=12] add_locality_zone success(locality_zone={tenant_id:1006, region_priority:0}) [2025-03-17 09:57:02.820918] INFO [SHARE] load_region (ob_locality_table_operator.cpp:373) [117552][LocltyRefTask][T0][Y0-0000000000000000-0-0] [lt=6] load region(ret=0, locality_info={version:0, local_region:"sys_region", local_zone:"zone1", local_idc:"default_idc", local_zone_type:0, local_zone_status:2, locality_region_array:[{region:"sys_region", region_priority:18446744073709551615, zone_array:["zone1"]}], locality_zone_array:[{tenant_id:1, region_priority:18446744073709551615}, {tenant_id:1001, region_priority:0}, {tenant_id:1002, region_priority:0}, {tenant_id:1003, region_priority:0}, {tenant_id:1004, region_priority:0}, {tenant_id:1005, region_priority:0}, {tenant_id:1006, region_priority:0}]}) [2025-03-17 09:57:02.833496] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=14] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1004, server_version={version:{val:1742176622661033000, v:0}, total_part_count:2, valid_inner_part_count:1, valid_user_part_count:1}, server_version_delta=172453, in_cluster_service=true, cluster_version={val:1742176622159882002, v:0}, min_cluster_version={val:1742176622159882002, v:0}, max_cluster_version={val:1742176622159882002, v:0}, get_cluster_version_err=0, cluster_version_delta=673604, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=5759590, succ_cluster_heartbeat_count=1, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176143516287002, v:0}, local_cluster_delta=479317199, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:57:02.841769] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=9] ====== traversal_flush timer task ====== [2025-03-17 09:57:02.841954] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539206][T1004_TimerWK0][T1004][YB420A0B093E-00062FB958F5C03E-0-0] [lt=22] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:60379}, rec_scn:{val:1742148082680599002, v:0}}, {tablet_id:{id:50380}, rec_scn:{val:1742148082680599003, v:0}}, {tablet_id:{id:233}, rec_scn:{val:1742148082680599004, v:0}}, {tablet_id:{id:50114}, rec_scn:{val:1742148082680599005, v:0}}, {tablet_id:{id:50192}, rec_scn:{val:1742148082680599006, v:0}}, {tablet_id:{id:60259}, rec_scn:{val:1742148082680599008, v:0}}, {tablet_id:{id:188}, rec_scn:{val:1742148082680599009, v:0}}, {tablet_id:{id:224}, rec_scn:{val:1742148082680599010, v:0}}, {tablet_id:{id:308}, rec_scn:{val:1742148082680599012, v:0}}, {tablet_id:{id:369}, rec_scn:{val:1742148082680599013, v:0}}, {tablet_id:{id:493}, rec_scn:{val:1742148082680599014, v:0}}, {tablet_id:{id:101011}, rec_scn:{val:1742148082680599015, v:0}}, {tablet_id:{id:101109}, rec_scn:{val:1742148082680599017, v:0}}, {tablet_id:{id:284}, rec_scn:{val:1742148082680599018, v:0}}, {tablet_id:{id:50149}, rec_scn:{val:1742148082680599019, v:0}}, {tablet_id:{id:307}, rec_scn:{val:1742148083181692001, v:0}}, {tablet_id:{id:50240}, rec_scn:{val:1742148083181692002, v:0}}, {tablet_id:{id:50355}, rec_scn:{val:1742148083181692003, v:0}}, {tablet_id:{id:182}, rec_scn:{val:1742148083181692004, v:0}}, {tablet_id:{id:101098}, rec_scn:{val:1742148083181692005, v:0}}, {tablet_id:{id:419}, rec_scn:{val:1742148083181692007, v:0}}, {tablet_id:{id:60305}, rec_scn:{val:1742148083181692008, v:0}}, {tablet_id:{id:60297}, rec_scn:{val:1742148083181692009, v:0}}, {tablet_id:{id:378}, rec_scn:{val:1742148083181692010, v:0}}, {tablet_id:{id:101036}, rec_scn:{val:1742148083181692011, v:0}}, {tablet_id:{id:60005}, rec_scn:{val:1742148083181692012, v:0}}, {tablet_id:{id:60130}, rec_scn:{val:1742148083181692013, v:0}}, {tablet_id:{id:101064}, rec_scn:{val:1742148083181692015, v:0}}, {tablet_id:{id:101080}, rec_scn:{val:1742148083181692016, v:0}}, {tablet_id:{id:50424}, rec_scn:{val:1742148083181692017, v:0}}, {tablet_id:{id:197}, rec_scn:{val:1742148083181692018, v:0}}, {tablet_id:{id:50381}, rec_scn:{val:1742148083181692019, v:0}}, }, max_consequent_callbacked_scn={val:1742176622661033001, v:0}, *this={this:0x7f79ae2db150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148008655744000, v:0}, clog_base_lsn:{lsn:805257216}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176620157204000, v:0}, tablet_change_checkpoint_scn:{val:1741946943947202003, v:0}, all_id_meta:{id_meta:[{limited_id:1742176637787617000, latest_log_ts:{val:1742176621158150001, v:0}}, {limited_id:3000001, latest_log_ts:{val:1742036552349716002, v:0}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.842160] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539206][T1004_TimerWK0][T1004][YB420A0B093E-00062FB958F5C03E-0-0] [lt=116] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1001}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:32, ordered_tablets:{{tablet_id:{id:200048}, rec_scn:{val:1742148095278530005, v:0}}, {tablet_id:{id:200127}, rec_scn:{val:1742148095278530006, v:0}}, {tablet_id:{id:1152921504606847070}, rec_scn:{val:1742148095278530007, v:0}}, {tablet_id:{id:1152921504606847104}, rec_scn:{val:1742148095278530008, v:0}}, {tablet_id:{id:1152921504606847061}, rec_scn:{val:1742148095697591001, v:0}}, {tablet_id:{id:200124}, rec_scn:{val:1742148095697591003, v:0}}, {tablet_id:{id:1152921504606847115}, rec_scn:{val:1742148095697591004, v:0}}, {tablet_id:{id:1152921504606847037}, rec_scn:{val:1742148095697591005, v:0}}, {tablet_id:{id:200130}, rec_scn:{val:1742148095697591006, v:0}}, {tablet_id:{id:200052}, rec_scn:{val:1742148095697591007, v:0}}, {tablet_id:{id:200063}, rec_scn:{val:1742148095697591008, v:0}}, {tablet_id:{id:200031}, rec_scn:{val:1742148095697591009, v:0}}, {tablet_id:{id:1152921504606847065}, rec_scn:{val:1742148095697591011, v:0}}, {tablet_id:{id:1152921504606847013}, rec_scn:{val:1742148095697591012, v:0}}, {tablet_id:{id:1152921504606847055}, rec_scn:{val:1742148095697591013, v:0}}, {tablet_id:{id:1152921504606847006}, rec_scn:{val:1742148095697591014, v:0}}, {tablet_id:{id:200086}, rec_scn:{val:1742148095697591015, v:0}}, {tablet_id:{id:1152921504606847002}, rec_scn:{val:1742148095697591017, v:0}}, {tablet_id:{id:200136}, rec_scn:{val:1742148095697591018, v:0}}, {tablet_id:{id:1152921504606847046}, rec_scn:{val:1742148095697591020, v:0}}, {tablet_id:{id:1152921504606847099}, rec_scn:{val:1742148095697591021, v:0}}, {tablet_id:{id:200059}, rec_scn:{val:1742148096198618001, v:0}}, {tablet_id:{id:200145}, rec_scn:{val:1742148096198618003, v:0}}, {tablet_id:{id:200034}, rec_scn:{val:1742148096198618005, v:0}}, {tablet_id:{id:200122}, rec_scn:{val:1742148096198618006, v:0}}, {tablet_id:{id:1152921504606847049}, rec_scn:{val:1742148096198618007, v:0}}, {tablet_id:{id:1152921504606847057}, rec_scn:{val:1742148096198618008, v:0}}, {tablet_id:{id:1152921504606847068}, rec_scn:{val:1742148096698960001, v:0}}, {tablet_id:{id:1152921504606847105}, rec_scn:{val:1742148096698960003, v:0}}, {tablet_id:{id:200025}, rec_scn:{val:1742148096698960004, v:0}}, {tablet_id:{id:1152921504606847101}, rec_scn:{val:1742148096698960005, v:0}}, {tablet_id:{id:1152921504606847036}, rec_scn:{val:1742148096698960006, v:0}}, }, max_consequent_callbacked_scn={val:1742176622661033001, v:0}, *this={this:0x7f79e6d15150, is_inited:true, freezing_scn:{val:1742148003844295000, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1004, ls_id:{id:1001}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742148081676232000, v:0}, clog_base_lsn:{lsn:4093390848}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:1742176620157204000, v:0}, tablet_change_checkpoint_scn:{val:1742008506458625001, v:0}, all_id_meta:{id_meta:[{limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}, {limited_id:1, latest_log_ts:{val:18446744073709551615, v:3}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79c85de030, is_in_stop_state_:false, is_inited_:true, id_:1001}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1001, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:02.842237] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [3539206][T1004_TimerWK0][T1004][YB420A0B093E-00062FB958F5C03E-0-0] [lt=75] succeed to traversal_flush(ret=0, ls_cnt=2) [2025-03-17 09:57:02.860244] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [3539234][T1004_LockWaitM][T1004][Y0-0000000000000000-0-0] [lt=29] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:57:02.861659] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=22] QSched: |cfg: root:1,x,0 net_in:1,1202M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:248K/59:16011 disk:248K/59:16011 t1m2:64K/16:16192 t1m2g0:64K/16:16192 t1004m2:92K/22:18120 t1004m2g0:92K/22:18120 t1003m2:92K/21:13663 t1003m2g0:92K/21:13663 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:57:02.864238] INFO [LIB] stat (utility.h:1210) [3539273][T1004_ApplySrv3][T1004][YB420A0B093E-00062FB8DBD9BB17-0-0] [lt=6] [APPLY STAT CB APPEND COST TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=19, this=0x7f79e6df6280, extra_info="") [2025-03-17 09:57:02.864272] INFO [LIB] stat (utility.h:1210) [3539273][T1004_ApplySrv3][T1004][YB420A0B093E-00062FB8DBD9BB17-0-0] [lt=33] [APPLY STAT CB IN QUEUE TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=15758, this=0x7f79e6df64b0, extra_info="") [2025-03-17 09:57:02.864283] INFO [LIB] stat (utility.h:1210) [3539273][T1004_ApplySrv3][T1004][YB420A0B093E-00062FB8DBD9BB17-0-0] [lt=10] [APPLY STAT CB WAIT COMMIT TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=0, this=0x7f79e6df66e0, extra_info="") [2025-03-17 09:57:02.864293] INFO [LIB] stat (utility.h:1210) [3539273][T1004_ApplySrv3][T1004][YB420A0B093E-00062FB8DBD9BB17-0-0] [lt=9] [APPLY STAT CB EXECUTE TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=-446, this=0x7f79e6df6910, extra_info="") [2025-03-17 09:57:02.864300] INFO [LIB] stat (utility.h:1210) [3539273][T1004_ApplySrv3][T1004][YB420A0B093E-00062FB8DBD9BB17-0-0] [lt=6] [APPLY STAT CB TOTAL TIME](cur_stat_count=27, stat_interval=5000000, avg (count/cost)=15331, this=0x7f79e6df6b40, extra_info="") [2025-03-17 09:57:02.871855] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=24] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=6) [2025-03-17 09:57:02.880281] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=3543) [2025-03-17 09:57:02.882719] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.898016] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1742176622897573000, start_id=1742176622897573000, end_id=1742176622897573001) [2025-03-17 09:57:02.913757] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=31] [LS_LOCATION]dump tenant ls location caches(tenant_id=1, tenant_ls_locations=[{cache_key:{tenant_id:1, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622104032, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:4}]}]) [2025-03-17 09:57:02.913791] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=33] [LS_LOCATION]dump tenant ls location caches(tenant_id=1001, tenant_ls_locations=[{cache_key:{tenant_id:1001, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622104941, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]}]) [2025-03-17 09:57:02.913801] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=9] [LS_LOCATION]dump tenant ls location caches(tenant_id=1002, tenant_ls_locations=[{cache_key:{tenant_id:1002, ls_id:{id:1002}, cluster_id:1741272511}, renew_time:1742176622107652, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}]}, {cache_key:{tenant_id:1002, ls_id:{id:1003}, cluster_id:1741272511}, renew_time:1742176622107654, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}, {cache_key:{tenant_id:1002, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622107649, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}]}, {cache_key:{tenant_id:1002, ls_id:{id:1001}, cluster_id:1741272511}, renew_time:1741767677708999, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}]}, {cache_key:{tenant_id:1002, ls_id:{id:1004}, cluster_id:1741272511}, renew_time:1741767677709054, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]}]) [2025-03-17 09:57:02.913824] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=22] [LS_LOCATION]dump tenant ls location caches(tenant_id=1003, tenant_ls_locations=[{cache_key:{tenant_id:1003, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622108569, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}]) [2025-03-17 09:57:02.913831] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=7] [LS_LOCATION]dump tenant ls location caches(tenant_id=1004, tenant_ls_locations=[{cache_key:{tenant_id:1004, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622109534, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}, {cache_key:{tenant_id:1004, ls_id:{id:1001}, cluster_id:1741272511}, renew_time:1742176622109536, replica_locations:[{server:"192.168.1.62:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}]) [2025-03-17 09:57:02.913841] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=9] [LS_LOCATION]dump tenant ls location caches(tenant_id=1005, tenant_ls_locations=[{cache_key:{tenant_id:1005, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622110523, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}]) [2025-03-17 09:57:02.913850] INFO [SHARE.LOCATION] dump_cache (ob_ls_location_service.cpp:1230) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E818-0-0] [lt=8] [LS_LOCATION]dump tenant ls location caches(tenant_id=1006, tenant_ls_locations=[{cache_key:{tenant_id:1006, ls_id:{id:1002}, cluster_id:1741272511}, renew_time:1742176622112888, replica_locations:[{server:"192.168.1.64:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:2}]}, {cache_key:{tenant_id:1006, ls_id:{id:1001}, cluster_id:1741272511}, renew_time:1742176622112886, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}, {cache_key:{tenant_id:1006, ls_id:{id:1003}, cluster_id:1741272511}, renew_time:1742176622112890, replica_locations:[{server:"192.168.1.65:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:3}]}, {cache_key:{tenant_id:1006, ls_id:{id:1}, cluster_id:1741272511}, renew_time:1742176622112883, replica_locations:[{server:"192.168.1.63:2882", role:1, sql_port:2881, replica_type:0, property:{memstore_percent_:100}, restore_status:{status:0}, proposal_id:1}]}]) [2025-03-17 09:57:02.915955] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859722847}], scn_array=[cnt:1, {val:1742176622897573000, v:0}], curr_size=588, accum_size=2464, time_cost=17841) [2025-03-17 09:57:02.933805] WDIAG [RPC.OBRPC] rpc_call (ob_rpc_proxy.ipp:453) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=20][errcode=-4006] Rpc proxy not inited(ret=-4006) [2025-03-17 09:57:02.936799] WDIAG [SHARE] get_restore_shared_storage_limit (ob_backup_connectivity.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4018] no exist row(ret=-4018, sql=SELECT max_iops, max_bandwidth FROM __all_backup_storage_info WHERE tenant_id = 1004 AND dest_id = '1') [2025-03-17 09:57:02.940236] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=21][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940268] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=31][errcode=0] update shared device limit failed(ret=0, i=2, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940277] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=8][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940283] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=5][errcode=0] update shared device limit failed(ret=0, i=5, limit.array_.at(i)={key_:{storage_id_:1001, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940291] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=8][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940299] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=7][errcode=0] update shared device limit failed(ret=0, i=8, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940306] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=7][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940316] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=10][errcode=0] update shared device limit failed(ret=0, i=11, limit.array_.at(i)={key_:{storage_id_:1002, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940329] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940340] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=0] update shared device limit failed(ret=0, i=14, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940363] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=21][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940361] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=15] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312807607526, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:02.940375] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=12][errcode=0] update shared device limit failed(ret=0, i=17, limit.array_.at(i)={key_:{storage_id_:1, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940386] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=10][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940397] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=0] update shared device limit failed(ret=0, i=20, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940414] INFO [LIB] do_thread_recycle (threads.cpp:163) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=24] recycle one thread(this=0x7f7b319af170, total=3, remain=2) [2025-03-17 09:57:02.940412] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940434] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=20] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312807607526, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:02.940437] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=25][errcode=0] update shared device limit failed(ret=0, i=23, limit.array_.at(i)={key_:{storage_id_:1003, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940448] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=9][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940458] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=9][errcode=0] update shared device limit failed(ret=0, i=26, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940469] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=10][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940481] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=12][errcode=0] update shared device limit failed(ret=0, i=29, limit.array_.at(i)={key_:{storage_id_:1004, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940498] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=15][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940509] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=0] update shared device limit failed(ret=0, i=32, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940521] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940532] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=0] update shared device limit failed(ret=0, i=35, limit.array_.at(i)={key_:{storage_id_:1005, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940545] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=12][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940557] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=12][errcode=0] update shared device limit failed(ret=0, i=38, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:2, value_:0}) [2025-03-17 09:57:02.940569] WDIAG [COMMON] update_limit (ob_io_manager.cpp:247) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=11][errcode=-2] update limit failed(ret=-2, limit={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}, limit_ids_[static_cast(limit.type_)]=-1) [2025-03-17 09:57:02.940579] WDIAG [COMMON] set_limit_v2 (ob_io_manager.cpp:535) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB95027E8DD-0-0] [lt=10][errcode=0] update shared device limit failed(ret=0, i=41, limit.array_.at(i)={key_:{storage_id_:1006, tenant_id:1004, category:1}, type_:5, value_:0}) [2025-03-17 09:57:02.940711] INFO pn_ratelimit (group.c:653) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=21] PNIO set ratelimit as 9223372036854775807 bytes/s, grp_id=2 [2025-03-17 09:57:02.940736] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117407][TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=20] trace max work area(auto_calc=true, tenant_max_memory_limit=1717986918, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=85899345, max_wa_memory_size=85899345, tmp_max_wa_memory_size=85899345, pre_mem_target=85899345, remain_memory_size=85899345, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=1441533952, washable_size=-2, max_workarea_memory_size=85899345, max_tenant_memory_size=276452966, wash_ratio=6, tenant_id=1003) [2025-03-17 09:57:02.940759] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:57:02.940787] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:57:02.940794] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117407][TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=30] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=10737418, calc_global_bound_size=1048576000000, mem_target_=85899345, wa_max_memory_size=85899345, profile_cnt_=0, total_memory_size=0, max_wa_size=85899345, avg_bound_size=10737418, max_bound_size=10737418, min_bound_size_=1048576) [2025-03-17 09:57:02.940808] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117407][TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, global_bound_size_=10737418, manual_calc_cnt_=44644, drift_size_=0, pre_drift_size=0, wa_max_memory_size=85899345, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1003, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=10737418, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=85899345, auto_calc=true, sql_mem_callback_.get_total_dump_size()=0) [2025-03-17 09:57:02.940846] INFO [SQL.ENG] get_max_work_area_size (ob_tenant_sql_memory_manager.cpp:944) [117407][TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=10] trace max work area(auto_calc=true, tenant_max_memory_limit=15461882266, total_alloc_size=0, tenant_work_area_memory_hold=0, tenant_work_area_max_size=773094113, max_wa_memory_size=773094113, tmp_max_wa_memory_size=773094113, pre_mem_target=773094113, remain_memory_size=773094113, ratio=-nan, alloc_ratio=0.000000000000000000e+00, hold_ratio=0.000000000000000000e+00, tenant_memory_hold=7393341440, washable_size=-2, max_workarea_memory_size=773094113, max_tenant_memory_size=8068540826, wash_ratio=6, tenant_id=1004) [2025-03-17 09:57:02.940884] INFO [SQL.ENG] calculate_global_bound_size (ob_tenant_sql_memory_manager.cpp:310) [117407][TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=12] timer to calc global bound size(ret=0, best_interval_idx=1099, global_bound_size_=96636764, calc_global_bound_size=1048576000000, mem_target_=773094113, wa_max_memory_size=773094113, profile_cnt_=0, total_memory_size=0, max_wa_size=773094113, avg_bound_size=96636764, max_bound_size=96636764, min_bound_size_=1048576) [2025-03-17 09:57:02.940892] INFO [SQL.ENG] calculate_global_bound_size_by_interval_info (ob_tenant_sql_memory_manager.cpp:1192) [117407][TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=8] timer to calc global bound size(ret=0, global_bound_size_=96636764, manual_calc_cnt_=492, drift_size_=0, pre_drift_size=0, wa_max_memory_size=773094113, sql_mem_callback_.get_total_alloc_size()=0, tenant_id_=1004, profile_cnt_=0, pre_profile_cnt_=0, pre_profile_cnt=0, calc_info.get_global_bound_size()=96636764, total_memory_size=0, cur_profile_cnt=0, calc_info.get_mem_target()=773094113, auto_calc=true, sql_mem_callback_.get_total_dump_size()=69192448) [2025-03-17 09:57:02.948998] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:57:02.950005] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859723435}], scn_array=[cnt:1, {val:1742176622933804000, v:0}], curr_size=588, accum_size=3052, time_cost=15250) [2025-03-17 09:57:02.961590] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:57:02.961645] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:57:02.961663] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:57:02.969113] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=9] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:929, free_obj_hold_cnt:159, allocator used:4186624, allocator total:4439040}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:149, free_obj_hold_cnt:931, allocator used:70838280, allocator total:70838280}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():0, full allocator used:0, full allocator total:0}, tablets_mem=75277320, tablets_mem_limit=1288490180, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:33, free_obj_hold_cnt:50, allocator used:169984, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=1078) [2025-03-17 09:57:02.973012] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [117761][T1_Occam][T1][YB420A0B093E-00062FB860254125-0-0] [lt=25] ====== tenant freeze timer task ====== [2025-03-17 09:57:02.982544] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463547808}], scn_array=[cnt:1, {val:1742176622948639000, v:0}], curr_size=587, accum_size=3544, time_cost=33541) [2025-03-17 09:57:02.982783] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:02.982888] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872707402}], scn_array=[cnt:1, {val:1742176622661033003, v:0}], curr_size=122, accum_size=488, time_cost=21223) [2025-03-17 09:57:02.987017] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162420958}], scn_array=[cnt:1, {val:1742176622661033003, v:0}], curr_size=122, accum_size=732, time_cost=25315) [2025-03-17 09:57:02.987116] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=14] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=2, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=15327, avg_log_submit_to_slide_time=15343) [2025-03-17 09:57:02.987168] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=15] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=3059, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:03.011553] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2679) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=105] dump tenant info(tenant={id:1, tenant_meta:{unit:{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:4, max_cpu:4, memory_size:"5GB", log_disk_size:"8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:4, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:4, }}, mode:0, create_timestamp:1741318291145813, is_removed:false, hidden_sys_data_disk_config_size:0}, super_block:{tenant_id:1, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:false, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}, unit_min_cpu:"4.000000000000000000e+00", unit_max_cpu:"4.000000000000000000e+00", total_worker_cnt:132, min_worker_cnt:42, max_worker_cnt:150, stopped:0, worker_us:15693645625796, recv_hp_rpc_cnt:2643089, recv_np_rpc_cnt:30856356, recv_lp_rpc_cnt:0, recv_mysql_cnt:316711, recv_task_cnt:330967, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:3458684675, workers:42, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=330 cnt[3]=0 cnt[4]=0 cnt[5]=52406474 cnt[6]=1083 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:25, queue_size:0, recv_req_cnt:429736, min_worker_cnt:1, max_worker_cnt:1, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1741316880365672}{group_id:12, queue_size:0, recv_req_cnt:30, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1741767308754693}{group_id:10000, queue_size:0, recv_req_cnt:5217793, min_worker_cnt:40, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=2357519 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:40, nesting_worker_cnt:9, token_change:1741316903035917}{group_id:28, queue_size:0, recv_req_cnt:774, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=188 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:9, token_change:1742153654383091}{group_id:15, queue_size:0, recv_req_cnt:288, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:2, nesting_worker_cnt:0, token_change:1741322027912233}{group_id:1, queue_size:0, recv_req_cnt:2725, min_worker_cnt:8, max_worker_cnt:8, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1741319143915535}{group_id:10, queue_size:0, recv_req_cnt:3431706, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:4, nesting_worker_cnt:0, token_change:1741316879397961}{group_id:20, queue_size:0, recv_req_cnt:13, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1741767706762888}{group_id:19, queue_size:0, recv_req_cnt:693839, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1742176612980323}{group_id:2, queue_size:0, recv_req_cnt:3433149, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:4, nesting_worker_cnt:0, token_change:1741318423584152}{group_id:9, queue_size:0, recv_req_cnt:3076501, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:4, nesting_worker_cnt:0, token_change:1741318425667232}{group_id:27, queue_size:0, recv_req_cnt:2600123, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:4, nesting_worker_cnt:0, token_change:1741316881953407}{group_id:5, queue_size:0, recv_req_cnt:8271762, min_worker_cnt:4, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:4, nesting_worker_cnt:0, token_change:1741318427659945}, rpc_stat_info: pcode=0x51c:cnt=696 pcode=0x717:cnt=289 pcode=0x52b:cnt=225 pcode=0x710:cnt=172 pcode=0x40e:cnt=89, token_change_ts:1741316878878679, tenant_role:1}) [2025-03-17 09:57:03.012138] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1671) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=585] dump throttled time info(id_=1, throttled_time_log=group_id: 25, group: OBCG_HB_SERVICE, throttled_time: 0;group_id: 12, group: OBCG_MYSQL_LOGIN, throttled_time: 0;group_id: 28, group: OBCG_DBMS_SCHED_JOB, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 1, group: OBCG_CLOG, throttled_time: 0;group_id: 10, group: OBCG_LOC_CACHE, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 9, group: OBCG_DETECT_RS, throttled_time: 0;group_id: 27, group: OBCG_DTL, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;group_id: 10000, group: ocp_monitor_group, throttled_time: 0;tenant_id: 1, tenant_throttled_time: 0;) [2025-03-17 09:57:03.012607] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=10] sql audit evict mem start(request_manager_->get_tenant_id()=1004, evict_low_mem_level=278313880, evict_high_mem_level=371085173, size_used=131072, mem_used=372274176) [2025-03-17 09:57:03.012362] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2679) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=223] dump tenant info(tenant={id:508, tenant_meta:{unit:{tenant_id:508, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:5, max_cpu:5, memory_size:"1GB", log_disk_size:"2GB", data_disk_size:0, min_iops:10000, max_iops:50000, iops_weight:0, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:5, }}, mode:0, create_timestamp:1741316878746843, is_removed:false, hidden_sys_data_disk_config_size:0}, super_block:{tenant_id:508, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:true, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}, unit_min_cpu:"5.000000000000000000e+00", unit_max_cpu:"5.000000000000000000e+00", total_worker_cnt:60, min_worker_cnt:52, max_worker_cnt:150, stopped:0, worker_us:9158945500, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:3883287, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:4274991050, workers:52, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info: pcode=0x5c0:cnt=63, token_change_ts:1741316878755122, tenant_role:0}) [2025-03-17 09:57:03.012826] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1671) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=463] dump throttled time info(id_=508, throttled_time_log=tenant_id: 508, tenant_throttled_time: 0;) [2025-03-17 09:57:03.012834] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2679) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=8] dump tenant info(tenant={id:509, tenant_meta:{unit:{tenant_id:509, unit_id:1000, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1000, name:"virtual_tenant_unit", resource:{min_cpu:2.5, max_cpu:2.5, memory_size:"1GB", log_disk_size:"2GB", data_disk_size:0, min_iops:10000, max_iops:50000, iops_weight:0, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:2, }}, mode:0, create_timestamp:1741316878739913, is_removed:false, hidden_sys_data_disk_config_size:0}, super_block:{tenant_id:509, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:true, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}, unit_min_cpu:"2.500000000000000000e+00", unit_max_cpu:"2.500000000000000000e+00", total_worker_cnt:35, min_worker_cnt:27, max_worker_cnt:150, stopped:0, worker_us:4579256016, recv_hp_rpc_cnt:0, recv_np_rpc_cnt:0, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:2134480134, workers:27, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:, rpc_stat_info:, token_change_ts:1741316878768639, tenant_role:0}) [2025-03-17 09:57:03.013290] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1671) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=455] dump throttled time info(id_=509, throttled_time_log=tenant_id: 509, tenant_throttled_time: 0;) [2025-03-17 09:57:03.013303] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2679) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=13] dump tenant info(tenant={id:1003, tenant_meta:{unit:{tenant_id:1003, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:1, max_cpu:1, memory_size:"1.6GB", log_disk_size:"4.8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}, super_block:{tenant_id:1003, replay_start_point:ObLogCursor{file_id=1, log_id=1, offset=0}, ls_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:false, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}, unit_min_cpu:"1.000000000000000000e+00", unit_max_cpu:"1.000000000000000000e+00", total_worker_cnt:25, min_worker_cnt:6, max_worker_cnt:150, stopped:0, worker_us:5223280658907, recv_hp_rpc_cnt:269, recv_np_rpc_cnt:785355, recv_lp_rpc_cnt:0, recv_mysql_cnt:0, recv_task_cnt:0, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:231896814, workers:6, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=3019932 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:28, queue_size:0, recv_req_cnt:519, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:9, token_change:1742114412895360}{group_id:20, queue_size:0, recv_req_cnt:846, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1741716064300723}{group_id:2, queue_size:0, recv_req_cnt:2322714, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1741595943687308}{group_id:5, queue_size:0, recv_req_cnt:62, min_worker_cnt:1, max_worker_cnt:150, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1741595946206237}, rpc_stat_info: pcode=0x524:cnt=97 pcode=0x150b:cnt=20 pcode=0x150a:cnt=20 pcode=0x745:cnt=9, token_change_ts:1741595954868416, tenant_role:1}) [2025-03-17 09:57:03.013933] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1671) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=630] dump throttled time info(id_=1003, throttled_time_log=group_id: 28, group: OBCG_DBMS_SCHED_JOB, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;tenant_id: 1003, tenant_throttled_time: 0;) [2025-03-17 09:57:03.014010] INFO [SERVER.OMT] run1 (ob_multi_tenant.cpp:2679) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=76] dump tenant info(tenant={id:1004, tenant_meta:{unit:{tenant_id:1004, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:8, max_cpu:8, memory_size:"14.4GB", log_disk_size:"43.2GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}, super_block:{tenant_id:1004, replay_start_point:ObLogCursor{file_id=5, log_id=1100611, offset=55124412}, ls_meta_entry:{[ver=1,mode=0,seq=39295][2nd=39296]}, tablet_meta_entry:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_hidden:false, version:4, snapshot_cnt:0, preallocated_seqs:{object_seq:60000, tmp_file_seq:60000, write_seq:60000}, auto_inc_ls_epoch:0, ls_cnt:0}, create_status:1, epoch:0}, unit_min_cpu:"8.000000000000000000e+00", unit_max_cpu:"8.000000000000000000e+00", total_worker_cnt:70, min_worker_cnt:34, max_worker_cnt:184, stopped:0, worker_us:5230786500150, recv_hp_rpc_cnt:1547, recv_np_rpc_cnt:59758, recv_lp_rpc_cnt:0, recv_mysql_cnt:195569, recv_task_cnt:1505, recv_large_req_cnt:0, tt_large_quries:0, pop_normal_cnt:1845762074, workers:34, nesting workers:8, req_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 , multi_level_queue:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=26302 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , group_map:{group_id:28, queue_size:0, recv_req_cnt:665, min_worker_cnt:8, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:9, token_change:1742176524442983}{group_id:18, queue_size:0, recv_req_cnt:1073925, min_worker_cnt:1, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:1, nesting_worker_cnt:0, token_change:1742176576699413}{group_id:15, queue_size:0, recv_req_cnt:131, min_worker_cnt:2, max_worker_cnt:2, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:2, nesting_worker_cnt:0, token_change:1741599227953535}{group_id:20, queue_size:0, recv_req_cnt:2265, min_worker_cnt:8, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1742148063844932}{group_id:19, queue_size:0, recv_req_cnt:23, min_worker_cnt:8, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:0, nesting_worker_cnt:0, token_change:1742011754361342}{group_id:2, queue_size:0, recv_req_cnt:4645250, min_worker_cnt:8, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:8, nesting_worker_cnt:0, token_change:1741595955417839}{group_id:5, queue_size:0, recv_req_cnt:722, min_worker_cnt:8, max_worker_cnt:184, multi_level_queue_:total_size=0 queue[0]=0 queue[1]=0 queue[2]=0 queue[3]=0 queue[4]=0 queue[5]=0 queue[6]=0 queue[7]=0 queue[8]=0 queue[9]=0 , recv_level_rpc_cnt:cnt[0]=0 cnt[1]=0 cnt[2]=0 cnt[3]=0 cnt[4]=0 cnt[5]=0 cnt[6]=0 cnt[7]=0 cnt[8]=0 cnt[9]=0 , worker_cnt:8, nesting_worker_cnt:0, token_change:1741595977793014}, rpc_stat_info: pcode=0x150a:cnt=40 pcode=0x150b:cnt=40 pcode=0x743:cnt=18 pcode=0x524:cnt=5, token_change_ts:1741595943076164, tenant_role:1}) [2025-03-17 09:57:03.014515] INFO [SERVER.OMT] print_throttled_time (ob_tenant.cpp:1671) [117689][MultiTenant][T0][Y0-0000000000000000-0-0] [lt=505] dump throttled time info(id_=1004, throttled_time_log=group_id: 28, group: OBCG_DBMS_SCHED_JOB, throttled_time: 0;group_id: 18, group: OBCG_DBA_COMMAND, throttled_time: 0;group_id: 15, group: OBCG_WR, throttled_time: 0;group_id: 20, group: OBCG_LOCK, throttled_time: 0;group_id: 19, group: OBCG_STORAGE, throttled_time: 0;group_id: 2, group: OBCG_ELECTION, throttled_time: 0;group_id: 5, group: OBCG_ID_SERVICE, throttled_time: 0;tenant_id: 1004, tenant_throttled_time: 0;) [2025-03-17 09:57:03.018272] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=13] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40365935820, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:03.022418] INFO [SERVER] release_record (ob_dl_queue.cpp:338) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=24] release ref is not 0 or release_wait_times < 3(rq_release_idx=234, ret=0, release_wait_times_=1, need_clean=false) [2025-03-17 09:57:03.022462] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=43] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=589824, evict_batch_count=1, elapse_time=9856, size_used=131072, mem_used=241250304) [2025-03-17 09:57:03.022505] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=6] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:03.031160] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA588-0-0] [lt=1][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:03.033662] INFO [STORAGE.TRANS] generate_new_version (ob_tenant_weak_read_server_version_mgr.cpp:120) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=58] [WRS] update tenant weak read server version(tenant_id=1004, server_version={version:{val:1742176622661033002, v:0}, total_part_count:2, valid_inner_part_count:1, valid_user_part_count:1, epoch_tstamp:1742176623033618}, version_delta=372627) [2025-03-17 09:57:03.057998] INFO [STORAGE.COMPACTION] runTimerTask (ob_tenant_tablet_scheduler_task_mgr.cpp:143) [117783][T1_TimerWK2][T1][YB420A0B093E-00062FB91AF8D16D-0-0] [lt=43] InfoPoolResizeTask(cost_ts=7) [2025-03-17 09:57:03.061684] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:03.064223] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=54] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:03.064363] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=11] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:03.071947] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=30] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=5) [2025-03-17 09:57:03.073811] INFO [CLOG] is_in_sync (ob_log_handler.cpp:709) [117882][T1_L5_G0][T1][YB420A0B0940-00063020D00CE199-0-0] [lt=28] is_in_sync(ret=0, id=1, is_log_sync=true, is_need_rebuild=false) [2025-03-17 09:57:03.080418] INFO [LIB] stat (utility.h:1210) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=34] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:57:03.082855] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.083291] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16716, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:57:03.083316] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872707524}], scn_array=[cnt:1, {val:1742176622661033004, v:0}], curr_size=122, accum_size=610, time_cost=21617) [2025-03-17 09:57:03.083354] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=23, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:57:03.083392] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=6, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:57:03.083416] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16825, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:57:03.083463] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=37] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:57:03.083783] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421080}], scn_array=[cnt:1, {val:1742176622661033004, v:0}], curr_size=122, accum_size=854, time_cost=22058) [2025-03-17 09:57:03.090046] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=36] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16345, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:57:03.090081] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=3666) [2025-03-17 09:57:03.090101] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=15] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=25, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:57:03.090118] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=3666) [2025-03-17 09:57:03.090166] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=20] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:57:03.090179] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=16457, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:57:03.090196] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=39, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:57:03.091900] WDIAG [SHARE] inner_get_ls_status_ (ob_ls_status_operator.cpp:955) [3539214][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=15][errcode=-4018] ls not exist in __all_ls_status table(ret=-4018, ret="OB_ENTRY_NOT_EXIST") [2025-03-17 09:57:03.091974] WDIAG [SHARE] get_duplicate_ls_status_info (ob_ls_status_operator.cpp:674) [3539214][T1004_DupTbLeas][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4018] fail to inner get ls status info(ret=-4018, ret="OB_ENTRY_NOT_EXIST", sql=SELECT * FROM __all_ls_status where tenant_id = 1004 and flag like "%DUPLICATE%", tenant_id=1004, exec_tenant_id=1003, need_member_list=false) [2025-03-17 09:57:03.097994] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=30] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=14, avg_log_batch_cnt=1, total_group_log_size=8460, avg_group_log_size=604) [2025-03-17 09:57:03.098023] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:57:03.118982] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=29] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:03.128904] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:03.128958] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:6, fail_bw:24537, [delay/us]:prepare:1, schedule:25, submit:1, rt:2742, total:2770, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:03.128983] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4505, iops:10, bw:44984, [delay/us]:prepare:5, schedule:19, submit:654, rt:16359, total:17037) [2025-03-17 09:57:03.128998] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=2, iops=2, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:03.129057] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=54] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:6, avg_bw:24536, [delay/us]: prepare:1, schedule:25, submit:1, device:2742, total:2770) [2025-03-17 09:57:03.129080] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4506, avg_iops:10, avg_bw:44983, [delay/us]: prepare:5, schedule:19, submit:654, device:16359, total:17037) [2025-03-17 09:57:03.129096] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:57:03.129138] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=18] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32714, [delay/us]:prepare:1, schedule:20, submit:1, rt:1463, total:1486, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:03.129172] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:8192 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4726, iops:13, bw:61339, [delay/us]:prepare:6, schedule:22, submit:274, rt:22317, total:22621) [2025-03-17 09:57:03.129185] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:03.129223] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=35] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32713, [delay/us]: prepare:1, schedule:20, submit:1, device:1463, total:1486) [2025-03-17 09:57:03.129253] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=20] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4726, avg_iops:13, avg_bw:61335, [delay/us]: prepare:6, schedule:22, submit:274, device:22317, total:22621) [2025-03-17 09:57:03.129269] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:57:03.129296] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8178, [delay/us]:prepare:5, schedule:15, submit:1, rt:18055, total:18076) [2025-03-17 09:57:03.129324] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4291, iops:21, bw:89958, [delay/us]:prepare:6, schedule:28, submit:1469, rt:14187, total:15691) [2025-03-17 09:57:03.129336] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=5, iops=5, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:03.129375] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=35] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8177, [delay/us]: prepare:5, schedule:15, submit:1, device:18055, total:18076) [2025-03-17 09:57:03.129395] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4291, avg_iops:21, avg_bw:89954, [delay/us]: prepare:6, schedule:28, submit:1469, device:14187, total:15691) [2025-03-17 09:57:03.129411] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:57:03.129453] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4088, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:3242, total:3249) [2025-03-17 09:57:03.129494] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:03.129528] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=30] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4088, [delay/us]: prepare:0, schedule:0, submit:600000000, device:3242, total:3249) [2025-03-17 09:57:03.129551] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:57:03.129570] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=13] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:03.129584] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129596] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129607] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129619] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129629] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129641] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129652] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129665] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:03.129675] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129685] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129700] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129710] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:03.129726] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS SERVER] net_in=47796kB/s, net_out=290kB/s, bucket_in=0kB/s, bucket_out=0kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:57:03.130461] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=30] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=29116416) [2025-03-17 09:57:03.130550] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=38] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=90, size_used=0, mem_used=0) [2025-03-17 09:57:03.130582] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=7] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:03.132977] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859724023}], scn_array=[cnt:1, {val:1742176623097302000, v:0}], curr_size=588, accum_size=3640, time_cost=34971) [2025-03-17 09:57:03.135847] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539253][T1004_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=141] get number(ret=0, service_type_=0, range=1, base_id=1742176623135602000, start_id=1742176623135602000, end_id=1742176623135602001) [2025-03-17 09:57:03.150302] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859724611}], scn_array=[cnt:1, {val:1742176623135602000, v:0}], curr_size=588, accum_size=4228, time_cost=14320) [2025-03-17 09:57:03.155356] INFO [SERVER] fill_tenant_ids_ (ob_all_virtual_tx_stat.cpp:141) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5721-0-0] [lt=9] succeed to get tenant ids(all_tenants_=[1, 1003, 1004]) [2025-03-17 09:57:03.156156] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:451) [3539236][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:57:03.156262] WDIAG read_multi_row (ob_table_access_helper.h:238) [3539236][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1004, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:57:03.157405] INFO delete_row (ob_table_access_helper.h:339) [3539236][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=18] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1004, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1742176623156281) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1742176623156281) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2025-03-17 09:57:03.157543] INFO [STORAGE.TABLELOCK] garbage_collect_for_all_ls_ (ob_table_lock_service.cpp:349) [3539236][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=37] finish check and clear obj lock(ls->get_ls_id()={id:1001}) [2025-03-17 09:57:03.161644] INFO [STORAGE.TRANS] refresh_gts (ob_gts_source.cpp:520) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=35] refresh gts(ret=0, ret="OB_SUCCESS", tenant_id=1, need_refresh=false, gts_local_cache={srr:[mts=1742176623161623], gts:1742176623161152000, latest_srr:[mts=1742176623161623]}) [2025-03-17 09:57:03.161676] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=31] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:57:03.162330] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=14][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463548517}], scn_array=[cnt:1, {val:1742176623148289000, v:0}], curr_size=587, accum_size=587, time_cost=12964) [2025-03-17 09:57:03.174854] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=32] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:57:03.182154] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=27] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:57:03.182923] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.183332] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539162][T1004_Occam][T1004][YB420A0B093E-00062FB7FA8C57C2-0-0] [lt=30] for each ls(succ_num=2, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:03.183976] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539188][T1004_Occam][T1004][YB420A0B093E-00062FB86F654125-0-0] [lt=20] ====== tenant freeze timer task ====== [2025-03-17 09:57:03.184874] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421202}], scn_array=[cnt:1, {val:1742176622661033005, v:0}], curr_size=122, accum_size=976, time_cost=23086) [2025-03-17 09:57:03.186788] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539162][T1004_Occam][T1004][YB420A0B093E-00062FB7FA8C57C4-0-0] [lt=27] for each ls(succ_num=2, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:03.198309] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [117806][T1_LockWaitMgr][T1][Y0-0000000000000000-0-0] [lt=21] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:57:03.206724] WDIAG read_and_convert_to_tuples_ (ob_table_access_helper.h:451) [3539418][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=33][errcode=-4008] iter failed(iter_times=0, ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:57:03.206802] WDIAG read_multi_row (ob_table_access_helper.h:238) [3539418][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=25][errcode=-4008] fail to read and convert to tuples(ret=-4008, ret="OB_ITER_END", MTL_ID()=1003, table=oceanbase.__all_detect_lock_info, condition=WHERE detect_func_no = 1 GROUP BY owner_id) [2025-03-17 09:57:03.207772] INFO delete_row (ob_table_access_helper.h:339) [3539418][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=11] GCTX.sql_proxy_ insert row success(ret=0, ret="OB_SUCCESS", MTL_ID()=1003, table=oceanbase.__all_dbms_lock_allocated, value=expiration <= usec_to_time(1742176623206813) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10, sql=DELETE FROM oceanbase.__all_dbms_lock_allocated WHERE expiration <= usec_to_time(1742176623206813) AND lockid NOT IN (SELECT obj_id FROM oceanbase.__all_detect_lock_info where obj_type = 13 or obj_type = 11)LIMIT 10) [2025-03-17 09:57:03.212858] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=8] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:57:03.212887] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=22] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=8594) [2025-03-17 09:57:03.218203] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [3539374][T1003_Occam][T1003][YB420A0B093E-00062FB804154126-0-0] [lt=21] ====== tenant freeze timer task ====== [2025-03-17 09:57:03.218655] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=9] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40430947532, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:03.219310] INFO [COMMON] wash (ob_kvcache_store.cpp:356) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=30] Wash time detail, (compute_wash_size_time=145, refresh_score_time=421, wash_time=0) [2025-03-17 09:57:03.221652] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=26] PNIO [ratelimit] time: 1742176623221650, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007646, add_bytes: 0 [2025-03-17 09:57:03.239194] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117882][T1_L5_G0][T1][YB420A0B0940-00063023B53C2B56-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1742176623238875000, start_id=1742176623238875000, end_id=1742176623238875001) [2025-03-17 09:57:03.246857] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:03.246891] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:03.247233] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=17][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176621158150000, replayable_scn:1742176621158150000, readable_scn:1742176621158150000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176622295227000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:03.247273] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176621158150000, replayable_scn:1742176621158150000, readable_scn:1742176621158150000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176622295227000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:03.247291] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176621158150000, replayable_scn:1742176621158150000, readable_scn:1742176621158150000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176622295227000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:03.247343] WDIAG [STORAGE.TRANS] inner_lock_for_read (ob_tx_data_functor.cpp:342) [117894][T1_L0_G0][T1][YB420A0B0941-00063020CFCD27C3-0-0] [lt=63][errcode=-6004] lock_for_read need retry(ret=-6004, tx_data={tx_id:{txid:14855963}, ref_cnt:1, state:"RUNNING", commit_version:{val:1742176623238875000, v:0}, start_scn:{val:1742176623238875000, v:0}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}, lock_for_read_arg_={mvcc_acc_ctx_:{type:1, abs_lock_timeout_ts:1742176653145990, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176623238875000, v:0}, tx_id:{txid:0}, scn:0}, tx_table_guards:{tx_table_guard:{tx_table:0x7f7ac1df1b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:0}, tx_desc:NULL, tx_ctx:null, mem_ctx:null, tx_scn:0, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, data_trans_id_:{txid:14855963}, data_sql_sequence_:{branch:0, seq:3}, read_latest_:false, read_uncommitted_:false, scn_:{val:1742176623238875000, v:0}}, tx_cc_ctx={state:10, prepare_version:{val:1742176623238875000, v:0}}) [2025-03-17 09:57:03.248370] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFC1-0-0] [lt=10][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:03.248406] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:03.248415] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:57:03.259453] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463549104}], scn_array=[cnt:1, {val:1742176623238875000, v:0}], curr_size=546, accum_size=1133, time_cost=20174) [2025-03-17 09:57:03.272065] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=23] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=4) [2025-03-17 09:57:03.272727] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] [COMPACTION DAG STATUS] (start_time=1742176613271102, end_time=1742176623272726, dag_status={}) [2025-03-17 09:57:03.272758] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=29] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272773] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=15] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272794] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=20] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272806] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272818] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272829] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272839] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272849] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272860] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:03.272870] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272884] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=13] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272896] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE/MEDIUM_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272908] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272916] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272924] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272931] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272939] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272947] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272954] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"VERIFY_CKM", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272961] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272968] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"UPDATE_SKIP_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272976] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"REFRESH_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272983] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272990] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.272998] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273005] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273021] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=15] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_DEL_LOB_META", dag_module_str:"DROP_VEC_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273028] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273036] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273044] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273051] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATE_WARMUP", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273059] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"WAIT_DATA_READY", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273066] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273075] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273082] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273089] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273097] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273105] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273113] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273120] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273127] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273135] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273152] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=16] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273159] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_CHECKE_CONVERT", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273166] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273173] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273181] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273188] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273196] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273203] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273211] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273218] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273225] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273234] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273239] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273244] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273249] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273256] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"INITIAL_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273263] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"START_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273271] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_TABLET_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273278] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"FINISH_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273286] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273293] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273300] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273308] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273315] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273322] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273329] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273336] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273344] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273351] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273358] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273366] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273373] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273388] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273399] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273407] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BUILD_TABLET_INFO", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273420] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273462] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=41] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273470] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:8, sys_task_type:24, dag_type_str:"VECTOR_INDEX_DAG", dag_module_str:"VECTOR_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:03.273478] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3635) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2025-03-17 09:57:03.273486] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2025-03-17 09:57:03.273494] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2025-03-17 09:57:03.273499] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2025-03-17 09:57:03.273505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2025-03-17 09:57:03.273509] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2025-03-17 09:57:03.273515] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2025-03-17 09:57:03.273521] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2025-03-17 09:57:03.273526] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2025-03-17 09:57:03.273532] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2025-03-17 09:57:03.273539] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4342) [117772][T1_DagScheduler][T1][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(dag_cnt=0, total_worker_cnt_=2, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2025-03-17 09:57:03.275190] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [3539247][T1004_LCLSender][T1004][Y0-0000000000000000-0-0] [lt=33] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f79c25fe2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:57:03.282666] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872707768}], scn_array=[cnt:1, {val:1742176623161152000, v:0}], curr_size=122, accum_size=854, time_cost=20811) [2025-03-17 09:57:03.283008] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.285585] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539350][T1003_Occam][T1003][YB420A0B093E-00062FB7FA9C57C2-0-0] [lt=19] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:03.285692] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [3539350][T1003_Occam][T1003][YB420A0B093E-00062FB7FA9C57C4-0-0] [lt=24] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:03.287336] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421324}], scn_array=[cnt:1, {val:1742176623161152000, v:0}], curr_size=122, accum_size=1098, time_cost=25486) [2025-03-17 09:57:03.296876] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=31] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8c88, ref_cnt_:1, raw_buf_:0x7f77a0cd8150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78627-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176623296249, enqueue_used:1, dequeue_used:29, submit_used:1, return_used:568, callback_enqueue_used:2, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:03.296934] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=55][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.296959] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=24][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.296978] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.296985] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8c88, ref_cnt_:1, raw_buf_:0x7f77a0cd8150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78627-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176623296249, enqueue_used:1, dequeue_used:29, submit_used:1, return_used:568, callback_enqueue_used:2, callback_dequeue_used:25, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.297023] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8c88, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78627-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176623296249, enqueue_used:1, dequeue_used:29, submit_used:1, return_used:568, callback_enqueue_used:2, callback_dequeue_used:25, callback_finish_used:-1, end_used:1742176623297018}}, part_id:-1}) [2025-03-17 09:57:03.297028] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=12][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.297406] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=33][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.297444] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=38][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.297456] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=11][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7ac862f510, src_buf_size=295) [2025-03-17 09:57:03.297467] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7698, ref_cnt_:1, raw_buf_:0x7f7ac862e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78627-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176623297070, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:299, callback_enqueue_used:1, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.297506] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=32][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c7698, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78627-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a41072080, time_log:{begin_ts:1742176623297070, enqueue_used:1, dequeue_used:19, submit_used:1, return_used:299, callback_enqueue_used:1, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176623297500}}, part_id:-1}) [2025-03-17 09:57:03.297568] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78627-0-0] [lt=1] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade956840, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222518824, assoc_session_id:3222518824, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623295698, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623293556, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ade0b21f8, id:3222518824, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade956840}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.298014] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=7872) [2025-03-17 09:57:03.314371] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=26] PNIO [ratelimit] time: 1742176623314370, bytes: 2535162511985, bw: 46.970831 MB/s, add_ts: 1001034, add_bytes: 49303413 [2025-03-17 09:57:03.326321] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7D-0-0] [lt=12] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:57:03.326349] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7D-0-0] [lt=26] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:57:03.331829] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=37][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859725199}], scn_array=[cnt:1, {val:1742176623297831000, v:0}], curr_size=588, accum_size=4816, time_cost=33806) [2025-03-17 09:57:03.348175] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:03.348226] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=28] gc stale ls task succ [2025-03-17 09:57:03.349139] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=24] get number(ret=0, service_type_=0, range=1, base_id=1742176623348986000, start_id=1742176623348986000, end_id=1742176623348986001) [2025-03-17 09:57:03.349812] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.351090] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859725787}], scn_array=[cnt:1, {val:1742176623334047000, v:0}], curr_size=588, accum_size=5404, time_cost=16180) [2025-03-17 09:57:03.351163] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222523028, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf325f40}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:03.351203] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=27] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:7}, to_seq_no={branch:0, seq:5}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634406} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:03.351227] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=20] end stmt(ret=0, tx_id=9634406, plain_select=false, stmt_type=2, savepoint={branch:0, seq:5}, tx_desc={this:0x7f7acf325f40, tx_id:{txid:9634406}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222523028, assoc_session_id:3222523028, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176623347923, active_ts:1742176623347923, commit_ts:-1, finish_ts:-1, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176653347916, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467463550344, first_scn:{branch:0, seq:4}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176623344735}], exec_info_reap_ts:1742176623344734, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623344730, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222523028, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf325f40}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.351333] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634406}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176653347916, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623348986}{self_ls_id:{id:1}, session_id:3222523028, associated_session_id:3222523028, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634406}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467463550344, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623348986, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634406} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.351398] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=57] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634406}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176653347916, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623348986}{self_ls_id:{id:1}, session_id:3222523028, associated_session_id:3222523028, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634406}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467463550344, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623348986, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634406} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.351461] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=60] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634406}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176623351330, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:03.351477] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=15] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf325f40, tx_id:{txid:9634406}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222523028, assoc_session_id:3222523028, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176623347923, active_ts:1742176623347923, commit_ts:-1, finish_ts:1742176623351104, timeout_us:29999993, lock_timeout_us:-1, expire_ts:1742176653347916, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467463550344, first_scn:{branch:0, seq:4}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176623344735}], exec_info_reap_ts:1742176623344734, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623344730, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:03.351549] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340E-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:03.357922] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=15] ====== check clog disk timer task ====== [2025-03-17 09:57:03.357963] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=34] get_disk_usage(ret=0, capacity(MB):=44236, used(MB):=4930) [2025-03-17 09:57:03.362882] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=57][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463549650}], scn_array=[cnt:1, {val:1742176623348986000, v:0}], curr_size=587, accum_size=1720, time_cost=13618) [2025-03-17 09:57:03.362972] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=10, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=17713, avg_log_submit_to_slide_time=17731) [2025-03-17 09:57:03.363006] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=25] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=3968, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:03.369952] INFO [STORAGE.TRANS] run1 (ob_xa_trans_heartbeat_worker.cpp:89) [117818][T1_ObXAHbWorker][T1][Y0-0000000000000000-0-0] [lt=29] XA scheduler heartbeat task statistics(avg_time=1) [2025-03-17 09:57:03.379976] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872707890}], scn_array=[cnt:1, {val:1742176623161152001, v:0}], curr_size=122, accum_size=976, time_cost=18079) [2025-03-17 09:57:03.380130] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=21][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.380186] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=55][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.380212] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=25][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.380224] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c88e0, ref_cnt_:1, raw_buf_:0x7f7aaac30150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063020D29B51D5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a96a16080, time_log:{begin_ts:1742176623377797, enqueue_used:1, dequeue_used:37, submit_used:1, return_used:2272, callback_enqueue_used:3, callback_dequeue_used:16, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.380294] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=60][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c88e0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063020D29B51D5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a96a16080, time_log:{begin_ts:1742176623377797, enqueue_used:1, dequeue_used:37, submit_used:1, return_used:2272, callback_enqueue_used:3, callback_dequeue_used:16, callback_finish_used:-1, end_used:1742176623380288}}, part_id:-1}) [2025-03-17 09:57:03.380300] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=32][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.380684] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=35][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.380711] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=27][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.380719] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a6033f510, src_buf_size=295) [2025-03-17 09:57:03.380729] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9270, ref_cnt_:1, raw_buf_:0x7f7a6033e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063020D29B51D5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a96a16080, time_log:{begin_ts:1742176623380352, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:304, callback_enqueue_used:1, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.380764] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=24][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9270, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063020D29B51D5-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a96a16080, time_log:{begin_ts:1742176623380352, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:304, callback_enqueue_used:1, callback_dequeue_used:8, callback_finish_used:-1, end_used:1742176623380755}}, part_id:-1}) [2025-03-17 09:57:03.380850] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B0940-00063020D29B51D5-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8ccec0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519134, assoc_session_id:3222519134, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623376831, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623376830, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222519134, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace8ccec0}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:03.381820] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=50] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16067, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:57:03.381846] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421446}], scn_array=[cnt:1, {val:1742176623161152001, v:0}], curr_size=122, accum_size=1220, time_cost=19922) [2025-03-17 09:57:03.381865] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:57:03.381883] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=22, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:57:03.381907] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:57:03.381932] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=13, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:57:03.381940] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=16163, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:57:03.381953] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=43, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:57:03.383079] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=21][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.391655] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=28][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:410, tid:117786}]) [2025-03-17 09:57:03.415184] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.415656] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=17] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.415675] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.415756] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.415817] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=14] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.415827] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=22] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.416182] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=42] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.418468] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=23] Slow write(duration=21513, write_len=4096) [2025-03-17 09:57:03.418490] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=21] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107789, offset=57288912}, end_cursor:ObLogCursor{file_id=5, log_id=1107790, offset=57289218}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17718050, len:1840, log_data_len:306, seq:1107789, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:03.418643] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=13] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:57:03.418677] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=20] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176623161152000, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176623161152000, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:03.419656] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=25] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40426753228, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:03.420863] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=13][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:03.421319] INFO [LIB] stat (utility.h:1210) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:57:03.432994] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859726375}], scn_array=[cnt:1, {val:1742176623420461000, v:0}], curr_size=1132, accum_size=6536, time_cost=11615) [2025-03-17 09:57:03.433230] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=42] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:03.451089] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Slow write(duration=17773, write_len=4096) [2025-03-17 09:57:03.451123] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=33] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107790, offset=57289218}, end_cursor:ObLogCursor{file_id=5, log_id=1107791, offset=57289515}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17720050, len:1534, log_data_len:297, seq:1107790, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:03.451289] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=28] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:57:03.451326] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731E-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176623161152001, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176623161152001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:03.451732] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:03.453563] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=0][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.453594] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=30][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.453610] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=15][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.453617] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9078, ref_cnt_:1, raw_buf_:0x7f7a22814150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453086, enqueue_used:1, dequeue_used:34, submit_used:1, return_used:423, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.453661] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=36][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9078, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453086, enqueue_used:1, dequeue_used:34, submit_used:1, return_used:423, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176623453656}}, part_id:-1}) [2025-03-17 09:57:03.453669] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=37][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453086, enqueue_used:1, dequeue_used:34, submit_used:1, return_used:423, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176623453656}}) [2025-03-17 09:57:03.453703] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=33][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:03.453717] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:03.454043] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.454079] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=35][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.454090] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f797409715a, src_buf_size=1611) [2025-03-17 09:57:03.454101] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=11][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9a08, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453727, enqueue_used:1, dequeue_used:45, submit_used:1, return_used:257, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.454157] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=36][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9a08, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF6340F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453727, enqueue_used:1, dequeue_used:45, submit_used:1, return_used:257, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176623454139}}, part_id:-1}) [2025-03-17 09:57:03.454162] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176623453727, enqueue_used:1, dequeue_used:45, submit_used:1, return_used:257, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176623454139}}) [2025-03-17 09:57:03.454194] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=32][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:03.454208] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.454234] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=25][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.454250] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=14][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:03.454264] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=14][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:03.454295] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=30][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:03.454363] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=67][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:57:03.454371] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:03.454379] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:03.454390] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:03.454396] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176653352964, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:03.454497] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=101][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176653352964, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 79DEC61A80300600, int: 1742176623451769}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}}) [2025-03-17 09:57:03.454553] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=54][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 79DEC61A80300600, int: 1742176623451769}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:57:03.454579] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=25][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:03.454588] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:57:03.454596] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}, ret=-4016) [2025-03-17 09:57:03.454629] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7aa426ae10, local_datums:0x7f7aa426aa60}, datums:[col_id=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 79DEC61A80300600, int: 1742176623451769}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8fd, hex: 79DEC61A80300600, int: 1742176623451769}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a905, hex: 79DEC61A80300600, int: 1742176623451769}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a90d, hex: 0800000000000000, int: 8}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a915, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7aa426a91d, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a923, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 28, flag: 0, null: 0, ptr: 0x7f7aa426a934, hex: 44424D535F53544154532E50555247455F5354415453284E554C4C29, cstr: DBMS_STATS.PURGE_STATS(NULL)}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a950, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a959, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a961, hex: 1227F4050D300600, int: 1741682352858898}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7aa426a969, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7aa426a97a, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176653352964, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176653352964, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176623452592000, v:0}, tx_id:{txid:9634411}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634411}, tx_desc:{this:0x7f7acf387140, tx_id:{txid:9634411}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520612, assoc_session_id:3222520612, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176623452592, active_ts:1742176623452592, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653452586, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467567306886, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176623451531}], exec_info_reap_ts:1742176623451530, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623451527, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:03.454721] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=90][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:57:03.454729] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=6][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 79DEC61A80300600, int: 1742176623451769}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 25, flag: 0, null: 0, ptr: 0x7f7aa426a8dc, hex: 4F50545F53544154535F484953544F52595F4D414E41474552, cstr: OPT_STATS_HISTORY_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7aa426a8f5, hex: 79DEC61A80300600, int: 1742176623451769}], store_rowkey:{"VARCHAR":"OPT_STATS_HISTORY_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:03.451769"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176653352964, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:57:03.454779] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=48][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:57:03.454786] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=6][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:57:03.454797] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:03.454804] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=6][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:57:03.454817] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:03.454824] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:57:03.454835] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:03.454842] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:57:03.454850] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=7][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176653452964, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56389156, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf387140, tx_id:{txid:9634411}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520612, assoc_session_id:3222520612, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176623452592, active_ts:1742176623452592, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653452586, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467567306886, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176623451531}], exec_info_reap_ts:1742176623451530, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623451527, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634411}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176653352964, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176653352964, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176653352964, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:57:03.455605] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=754][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:57:03.455619] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=13][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:03.455637] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=17][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:03.455650] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:57:03.455663] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:57:03.455676] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:57:03.455693] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=16][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:57:03.455702] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:03.455711] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=9][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:57:03.455719] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=8][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:57:03.455731] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=11][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:03.455742] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:03.455755] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}) [2025-03-17 09:57:03.455776] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=20][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:57:03.455793] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=15][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:03.455817] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=16] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222520612, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf387140}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:03.455867] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=32] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1066}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634411} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:03.455907] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=37] end stmt(ret=0, tx_id=9634411, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf387140, tx_id:{txid:9634411}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520612, assoc_session_id:3222520612, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176623452592, active_ts:1742176623452592, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653452586, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467567306886, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623451531}], exec_info_reap_ts:1742176623451530, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623451527, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222520612, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf387140}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.455968] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=60][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:03.455981] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=13][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:03.455993] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:03.456012] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=11][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:03.456037] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=24][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:57:03.456049] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER')) [2025-03-17 09:57:03.456071] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:57:03.456085] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=13][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), is_user_sql=false) [2025-03-17 09:57:03.456100] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=14][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176623451769), usec_to_time(1742176623451769), 8, usec_to_time(1742176623451769), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53544154532E50555247455F5354415453284E554C4C29', 'SCHEDULED', usec_to_time(1741682352858898), 'OPT_STATS_HISTORY_MANAGER'), conn=0x7f7ac70b23a0) [2025-03-17 09:57:03.456115] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=14][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:57:03.456213] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=12][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634411}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176653452586, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623452592}{self_ls_id:{id:1}, session_id:3222520612, associated_session_id:3222520612, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634411}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467567306886, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623452592, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634411} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.456333] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=105] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634411}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176653452586, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623452592}{self_ls_id:{id:1}, session_id:3222520612, associated_session_id:3222520612, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634411}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467567306886, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623452592, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634411} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.456445] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=107] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634411}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176623456209, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:03.456467] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=20] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf387140, tx_id:{txid:9634411}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222520612, assoc_session_id:3222520612, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176623452592, active_ts:1742176623452592, commit_ts:-1, finish_ts:1742176623455772, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653452586, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467567306886, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623451531}], exec_info_reap_ts:1742176623451530, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623451527, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:03.456570] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=41][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:57:03.456581] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:57:03.456593] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF6340F-0-0] [lt=10][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:03.462013] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117425][IO_SYNC_CH0][T0][Y0-0000000000000000-0-0] [lt=14] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:2, threads_idle_time:1312808283048, tenant_id:500}, cur_thread_count=2, cnt=1, new_thread_count=3) [2025-03-17 09:57:03.462241] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117425][IO_SYNC_CH0][T0][Y0-0000000000000000-0-0] [lt=30] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:2, threads_idle_time:1312808283048, tenant_id:500}, cur_thread_count=2, cnt=1, new_thread_count=3) [2025-03-17 09:57:03.464010] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859727507}], scn_array=[cnt:1, {val:1742176623452592000, v:0}], curr_size=749, accum_size=7285, time_cost=10958) [2025-03-17 09:57:03.466258] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=38] start do ls ha handler(ls_id_array_=[{id:1}, {id:1001}]) [2025-03-17 09:57:03.472173] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=28] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=3) [2025-03-17 09:57:03.480632] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:106) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=36] tx gc loop thread is running(MTL_ID()=1) [2025-03-17 09:57:03.480655] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:113) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=23] try gc retain ctx [2025-03-17 09:57:03.480662] INFO [STORAGE.TRANS] run1 (ob_tx_loop_worker.cpp:120) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=7] try to retry start_working [2025-03-17 09:57:03.480685] INFO [STORAGE] update_min_start_scn_info (ob_tx_table.cpp:1053) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=5] finish update min start scn(max_decided_scn={val:1742176623348986000, v:0}, ctx_min_start_scn_info_={min_start_scn_in_ctx_:{val:1742176618344925000, v:0}, keep_alive_scn_:{val:1742176618460984001, v:0}, update_ts_:1742176623480162}) [2025-03-17 09:57:03.481304] INFO [STORAGE.TRANS] do_tx_gc_ (ob_tx_loop_worker.cpp:271) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=13] [Tx Loop Worker] check tx scheduler success(MTL_ID()=1, *ls_ptr={running_state:{state:1}, ls_meta:{tenant_id:1, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742067167143792000, v:0}, clog_base_lsn:{lsn:4831543296}, rebuild_seq:1, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1742010174946113000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636588679000, latest_log_ts:{val:1742176619945902001, v:0}}, {limited_id:16000001, latest_log_ts:{val:1742139624749258001, v:0}}, {limited_id:220000001, latest_log_ts:{val:1742138690841156001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:4, palf_env_:0x7f7b067ea030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:100, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}) [2025-03-17 09:57:03.481402] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=66] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=3959, avg_group_log_size=395) [2025-03-17 09:57:03.483161] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:120) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=24] tenant info is invalid(ret=-4076, query_ts=0, standby_scn={val:0, v:0}, this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:57:03.483193] INFO [STORAGE.TRANS] query_and_update_last_id (ob_standby_timestamp_service.cpp:134) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=31] ObStandbyTimestampService state(*this={inited:true, last_id:-1, tenant_id:1004, epoch:-1, self:"192.168.1.62:2882", switch_to_leader_ts:-1}) [2025-03-17 09:57:03.483204] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.483738] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421568}], scn_array=[cnt:1, {val:1742176623161152002, v:0}], curr_size=122, accum_size=122, time_cost=21768) [2025-03-17 09:57:03.483909] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708012}], scn_array=[cnt:1, {val:1742176623161152002, v:0}], curr_size=122, accum_size=1098, time_cost=21951) [2025-03-17 09:57:03.522567] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=12] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=589824, evict_batch_count=0, elapse_time=1, size_used=131072, mem_used=241250304) [2025-03-17 09:57:03.522623] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=13] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:03.532645] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859728256}], scn_array=[cnt:1, {val:1742176623498181000, v:0}], curr_size=588, accum_size=7873, time_cost=34230) [2025-03-17 09:57:03.532879] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=27][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.532920] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=40][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.532941] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.532952] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c89b8, ref_cnt_:1, raw_buf_:0x7f7ac2552170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0941-00063020D34365F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623521409, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:11417, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.532998] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=39][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c89b8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0941-00063020D34365F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623521409, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:11417, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:-1, end_used:1742176623532992}}, part_id:-1}) [2025-03-17 09:57:03.533005] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=1][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623521409, enqueue_used:2, dequeue_used:22, submit_used:1, return_used:11417, callback_enqueue_used:2, callback_dequeue_used:23, callback_finish_used:151, end_used:-35}}) [2025-03-17 09:57:03.533051] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=43][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:03.533066] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=12][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:03.533434] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=29][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.533458] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=24][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.533465] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaacbf510, src_buf_size=295) [2025-03-17 09:57:03.533472] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c78d8, ref_cnt_:1, raw_buf_:0x7f7aaacbe150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0941-00063020D34365F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623533078, enqueue_used:1, dequeue_used:46, submit_used:2, return_used:275, callback_enqueue_used:1, callback_dequeue_used:29, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.533507] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=9][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623533078, enqueue_used:1, dequeue_used:46, submit_used:2, return_used:275, callback_enqueue_used:1, callback_dequeue_used:29, callback_finish_used:-1, end_used:1742176623533502}}) [2025-03-17 09:57:03.533504] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=29][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c78d8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0941-00063020D34365F9-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac6a9a080, time_log:{begin_ts:1742176623533078, enqueue_used:1, dequeue_used:46, submit_used:2, return_used:275, callback_enqueue_used:1, callback_dequeue_used:29, callback_finish_used:-1, end_used:1742176623533502}}, part_id:-1}) [2025-03-17 09:57:03.533528] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=19][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:03.533534] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=4][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac6a9a080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.533554] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=19][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac6a9a080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.533571] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=14][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:03.533585] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=13][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:03.533606] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=21][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f78178262f0, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f7a8d8839e0, last_rowkey:0x7f7a8d883b48, discrete_rowkey_array:0x7f7a8d883c68, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883a70, hex: EB03000000000000, int: 1003}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883aa8, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b08, hex: C8F2AD7DB79BD4E7, int: -1741595943495667000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a8d883b20, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f7a8d883963, ps_node_array:0x7f7a8d8839a0, data_buf:0x7f7a8d883963, data_buf_size:1031}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f7a8d883464, tablet_:0x7f7a8d8820a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:03.533651] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=44][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=2, prefetched_cnt=0) [2025-03-17 09:57:03.533657] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=5][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:03.533664] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:03.533676] WDIAG [STORAGE] inner_open (ob_sstable_multi_version_row_iterator.cpp:162) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:03.533683] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176653419319, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc58590, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176623505576000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7ac6b26090, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176623520588, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:03.533930] WDIAG [STORAGE] scan (ob_sstable.cpp:427) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=246][errcode=-4016] Fail to open row scanner(ret=-4016, param={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176653419319, ls_id:{id:1}, tablet_id:{id:345}, query_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, store_ctx:0x7f7b0cc58590, limit_param:null, stmt_allocator:0x7f7817811e08, allocator:0x7f7817811e40, range_allocator:null, table_scan_stat:0x7f7817813868, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:1741543202134877000, snapshot_version:1742176623505576000}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=1, used=808, total=7936, lob_locator_helper:{table_id:345, ls_id:1, tx_id:0, snapshot_version:0, tx_read_snapshot:{this:0x7f7ac6b26090, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, rowid_version:0, rowid_project_:[cnt:0], enable_locator_v2:true, is_inited:true, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tx_seq_base:1742176623520588, is_access_index:false}, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:0x7f7817817860, sample_filter:null, mview_scan_info:NULL}, key_range={start_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814610, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=1:MIN_OBJ], store_rowkey:{"BIGINT":1},MIN}, end_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 8, flag: 0, null: 0, ptr: 0x7f7817814690, hex: 0100000000000000, int: 1, num_digit0: 0}, idx=1:MAX_OBJ], store_rowkey:{"BIGINT":1},MAX}, group_idx:0, border_flag:{flag:0}}, *this={ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}) [2025-03-17 09:57:03.534329] WDIAG [STORAGE] construct_iters (ob_multiple_scan_merge.cpp:198) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=396][errcode=-4016] Fail to get iterator(ret=-4016, i=1, *iter_pram={table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7817812260, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:true, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7817814260}) [2025-03-17 09:57:03.534526] WDIAG [STORAGE] open (ob_multiple_scan_merge.cpp:60) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=196][errcode=-4016] fail to construct iters(ret=-4016) [2025-03-17 09:57:03.534533] WDIAG [STORAGE] init_and_open_scan_merge_iter_ (ob_table_scan_iterator.cpp:651) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] Fail to open multiple merge iterator(ret=-4016) [2025-03-17 09:57:03.534539] WDIAG [STORAGE] open_iter (ob_table_scan_iterator.cpp:558) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=5][errcode=-4016] init and open scan merge iterator failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:03.534548] WDIAG [STORAGE] init (ob_table_scan_iterator.cpp:355) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=8][errcode=-4016] fail to open iter(ret=-4016, cached_iter_node_=NULL, *this={}) [2025-03-17 09:57:03.534554] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2373) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=5][errcode=-4016] failed to init table scan iterator, (ret=-4016) [2025-03-17 09:57:03.534562] WDIAG [STORAGE] inner_table_scan (ob_ls_tablet_service.cpp:2379) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] failed to do table scan(ret=-4016, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176653419319, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176623520588, need_update_tablet_param:false}, *this={ObIRoleChangeSubHandler:"Dummy"}, data_max_schema_version=0) [2025-03-17 09:57:03.534769] WDIAG [STORAGE] table_scan (ob_ls_tablet_service.cpp:674) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=206][errcode=-4016] failed to do table scan(ret=-4016, &iter=0x7f7b0cc57910, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176653419319, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176623520588, need_update_tablet_param:false}) [2025-03-17 09:57:03.534981] WDIAG [STORAGE] table_scan (ob_access_service.cpp:371) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=211][errcode=-4016] Fail to scan table, (ret=-4016, ls_id={id:1}, param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176653419319, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176623520588, need_update_tablet_param:false}) [2025-03-17 09:57:03.535221] WDIAG [SQL.DAS] do_table_scan (ob_das_scan_iter.cpp:83) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=239][errcode=-4016] fail to scan table(scan_param={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176653419319, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:13, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:false, output_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], op_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], trans_desc:NULL, snapshot:{this:0x7f7817813f90, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176623520588, need_update_tablet_param:false}, ret=-4016) [2025-03-17 09:57:03.535406] WDIAG [SQL.DAS] open_op (ob_das_scan_op.cpp:569) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=184][errcode=-4016] iter tree failed to do table scan(ret=-4016) [2025-03-17 09:57:03.535411] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=5][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:03.535416] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:102) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=4][errcode=-4016] start das task failed(ret=-4016, *task_wrapper.at(i)={parent:{tenant_id:1, task_id:212097053, op_type:1, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7ace8b6aa0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222523305, assoc_session_id:3222523305, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623520589, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623520588, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, snapshot:{this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, tablet_id:{id:345}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:345}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:345, related_table_ids:[cnt:0], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:1}, in_retry:0, partition_id:345, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:0], related_rtdefs:[cnt:0], task_status:1, related_tablet_ids:[cnt:0], das_task_node:, plan_line_id:0}, scan_ctdef:{ObDASBaseCtDef:{op_type:1, children_cnt:0}, ref_table_id:345, access_column_ids:[cnt:13, 18, 19, 28, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27], aggregate_column_ids:[cnt:0], group_by_column_ids:[cnt:0], schema_version:1741278087142176, table_param:{table_id:345, output_projector:[cnt:13, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:13, true, true, true, true, true, true, true, true, true, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:14, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:13, 0, 1, 14, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13], data:0x7f7aad3ce1b0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:13, 0, 1, 12, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11], data:0x7f7aad3ce1f0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}}, cols_desc:{ObIArray:[cnt:13, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=28 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=21 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=22 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=24 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f7aad3ce110, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:13, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7aad3ce2a0, count:13, allocator:0x7f7ab01c26b0, capacity:13, init_cnt:13}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:2048, precision:-1, scale:-1}, orig_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, cur_default_value:{"VARCHAR":"", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:100, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:536870911, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:true, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, pd_expr_spec:{calc_exprs:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], access_exprs:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}], ext_file_column_exprs:[cnt:0], ext_column_convert_exprs:[cnt:0], max_batch_size:16, pushdown_filters:[cnt:1, {type:"T_OP_LIKE", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:1}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:203, eval_func:0x55c5caedc240, eval_batch_func:0x55c5caedc770, inner_functions:null, inner_func_cnt:0, arg_cnt:3, parent_cnt:0, frame_idx:2, datum_off:2600, res_buf_off:22207, dyn_buf_header_offset:7551, res_buf_len:8, eval_flags_off:5743, pvt_skip_off:2792, expr_ctx_id:0, extra:1, batch_idx_mask:18446744073709551615, vector_header_off:5217, eval_vector_func:0x55c5caedcc70, local_session_var_id:-1, this:0x7f7aad3cac48}], pd_storage_flag:{pd_flag:16}, trans_info_expr:NULL}, group_id_expr:NULL, result_output:[cnt:13, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:600, res_buf_off:9855, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5663, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4567, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca338}, {type:"T_REF_COLUMN", datum_meta:{type:"TIMESTAMP", cs_type:"binary", scale:6, precision:25}, obj_meta:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:195, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:800, res_buf_off:9983, dyn_buf_header_offset:6015, res_buf_len:8, eval_flags_off:5671, pvt_skip_off:992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4632, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca420}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:0, res_buf_off:7551, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5639, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4372, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:200, res_buf_off:7679, dyn_buf_header_offset:5759, res_buf_len:8, eval_flags_off:5647, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4437, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca168}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1000, res_buf_off:10111, dyn_buf_header_offset:6015, res_buf_len:128, eval_flags_off:5679, pvt_skip_off:1192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4697, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca508}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1200, res_buf_off:12159, dyn_buf_header_offset:6271, res_buf_len:128, eval_flags_off:5687, pvt_skip_off:1392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4762, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca5f0}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1400, res_buf_off:14207, dyn_buf_header_offset:6527, res_buf_len:100, eval_flags_off:5695, pvt_skip_off:1592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4827, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca6d8}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1600, res_buf_off:15807, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5703, pvt_skip_off:1792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4892, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca7c0}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:1800, res_buf_off:15935, dyn_buf_header_offset:6783, res_buf_len:8, eval_flags_off:5711, pvt_skip_off:1992, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4957, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca8a8}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2000, res_buf_off:16063, dyn_buf_header_offset:6783, res_buf_len:128, eval_flags_off:5719, pvt_skip_off:2192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5022, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca990}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2200, res_buf_off:18111, dyn_buf_header_offset:7039, res_buf_len:128, eval_flags_off:5727, pvt_skip_off:2392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5087, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3caa78}, {type:"T_REF_COLUMN", datum_meta:{type:"LONGTEXT", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"LONGTEXT", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:2, datum_off:2400, res_buf_off:20159, dyn_buf_header_offset:7295, res_buf_len:128, eval_flags_off:5735, pvt_skip_off:2592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:5152, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3cab60}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:3, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:2, datum_off:400, res_buf_off:7807, dyn_buf_header_offset:5759, res_buf_len:128, eval_flags_off:5655, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:4502, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aad3ca250}], is_get:false, is_external_table:false, external_files:[cnt:0], external_file_format_str:{str:""}, external_file_location:{str:""}, trans_info_expr:NULL, ir_scan_type:0, rowkey_exprs:[cnt:0], table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, doc_id_idx:-1, vec_vid_idx:-1, index_merge_idx:-1, pre_query_range:[], is_index_merge:0}, scan_rtdef:{ObDASBaseRtDef:{op_type:1, children_cnt:0}, tenant_schema_version:-1, limit_param:{offset_:0, limit_:-1}, need_scn:false, force_refresh_lc:false, frozen_version:-1, fb_snapshot:{val:18446744073709551615, v:3}, fb_read_tx_uncommitted:false, timeout_ts:1742176653419319, tx_lock_timeout:-1, sql_mode:4194304, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, tsc_monitor_info:{*io_read_bytes_:295*ssstore_read_bytes_:0*ssstore_read_row_cnt_:0*memstore_read_row_cnt_:0}, key_ranges:[], ss_key_ranges:[], mbr_filters:[], scan_op_id:18446744073709551615, scan_rows_size:-1, das_tasks_key:{fake_unique_id_:0, timestamp_:0}}, scan_range:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,({"BIGINT":1},MIN;{"BIGINT":1},MAX)"}], result:{}, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}}) [2025-03-17 09:57:03.535825] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=408][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:03.535835] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=9][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:03.535840] WDIAG [SQL.DAS] do_table_scan (ob_das_merge_iter.cpp:260) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=4][errcode=-4016] failed to execute all das task(ret=-4016) [2025-03-17 09:57:03.535844] WDIAG [SQL.ENG] do_table_scan (ob_table_scan_op.cpp:2699) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=4][errcode=-4016] execute all das scan task failed(ret=-4016) [2025-03-17 09:57:03.535851] WDIAG [SQL.ENG] do_init_before_get_row (ob_table_scan_op.cpp:1822) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6][errcode=-4016] fail to do table scan(ret=-4016) [2025-03-17 09:57:03.535858] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2588) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6][errcode=-4016] failed to init before get row(ret=-4016) [2025-03-17 09:57:03.535862] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=3][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:03.535867] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=3][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f781780bfb0}, id=0, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:03.535880] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=12][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:03.535887] WDIAG [SQL.EXE] sync_send_result (ob_remote_executor_processor.cpp:286) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:03.535895] WDIAG [SQL.EXE] execute_remote_plan (ob_remote_executor_processor.cpp:485) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=7][errcode=-4016] sync send result failed(ret=-4016) [2025-03-17 09:57:03.535915] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=6] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8b6aa0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222523305, assoc_session_id:3222523305, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623520589, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623520588, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222523305, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace8b6aa0}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:03.535952] WDIAG [SQL.EXE] execute_with_sql (ob_remote_executor_processor.cpp:716) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D34365F9-0-0] [lt=32][errcode=-4016] execute remote plan failed(ret=-4016, task={tenant_schema_version:1741661297536408, sys_schema_version:1741661297536408, runner_svr:"192.168.1.62:2882", ctrl_svr:"192.168.1.65:2882", task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.65:2882", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:18446744073709551615}, task_id:0, task_cnt:0}, remote_sql_info:{use_ps:true, is_batched_stmt:false, is_original_ps_mode:false, ps_param_cnt:2, remote_sql:"SELECT * FROM __all_ls_status where tenant_id = ? and flag like ?", ps_params:[{obj:{"BIGINT":1}, accuracy:{length:1, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}, {obj:{"VARCHAR":"%DUPLICATE%", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}, accuracy:{length:11, precision:1, scale:-1}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"COERCIBLE"}}]}, snapshot:{this:0x7f7817804208, valid:false, source:0, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}}, exec_ctx_.get_das_ctx().get_snapshot()={this:0x7f78178094a0, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}) [2025-03-17 09:57:03.539826] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463550350}], scn_array=[cnt:1, {val:1742176623505576000, v:0}], curr_size=546, accum_size=2379, time_cost=34183) [2025-03-17 09:57:03.543523] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=55] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8461) [2025-03-17 09:57:03.543552] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=29, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:57:03.543588] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=8, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:57:03.543610] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=21] [PALF STAT LOG LIFE TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=18199, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:57:03.543620] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=14, avg_log_gen_to_freeze_time=496, avg_log_gen_to_submit_time=508, avg_log_submit_to_flush_time=17676, avg_log_submit_to_slide_time=17691) [2025-03-17 09:57:03.543638] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=11] [PALF STAT FLUSH CB COST TIME](cur_stat_count=14, stat_interval=1000000, avg (count/cost)=37, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:57:03.543657] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8461, hit_cnt=15, read_cnt=15, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:03.544140] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=14] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:57:03.544177] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=31] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:57:03.544189] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=10] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=1, times=11749) [2025-03-17 09:57:03.548102] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=24][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.548143] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=40][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.548177] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=33][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.548187] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=9][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310ca308, ref_cnt_:1, raw_buf_:0x7f7ac862e150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0077-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623547708, enqueue_used:1, dequeue_used:42, submit_used:1, return_used:325, callback_enqueue_used:2, callback_dequeue_used:20, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.548244] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=43][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310ca308, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0077-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623547708, enqueue_used:1, dequeue_used:42, submit_used:1, return_used:325, callback_enqueue_used:2, callback_dequeue_used:20, callback_finish_used:-1, end_used:1742176623548233}}, part_id:-1}) [2025-03-17 09:57:03.548257] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=33][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623547708, enqueue_used:1, dequeue_used:42, submit_used:1, return_used:325, callback_enqueue_used:2, callback_dequeue_used:20, callback_finish_used:-1, end_used:1742176623548233}}) [2025-03-17 09:57:03.548290] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=30][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:03.548300] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=7][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:03.548614] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=43][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.548639] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=24][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.548646] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a41039510, src_buf_size=295) [2025-03-17 09:57:03.548653] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8808, ref_cnt_:1, raw_buf_:0x7f7a41038150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0077-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623548307, enqueue_used:1, dequeue_used:12, submit_used:1, return_used:271, callback_enqueue_used:1, callback_dequeue_used:19, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:03.548700] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=40][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8808, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0077-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623548307, enqueue_used:1, dequeue_used:12, submit_used:1, return_used:271, callback_enqueue_used:1, callback_dequeue_used:19, callback_finish_used:-1, end_used:1742176623548694}}, part_id:-1}) [2025-03-17 09:57:03.548728] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=6][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac2550080, time_log:{begin_ts:1742176623548307, enqueue_used:1, dequeue_used:12, submit_used:1, return_used:271, callback_enqueue_used:1, callback_dequeue_used:19, callback_finish_used:118, end_used:-36}}) [2025-03-17 09:57:03.548763] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=32][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:03.548775] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=10][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac2550080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.548795] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=20][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac2550080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:03.548814] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=16][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:03.548827] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=13][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc52090}) [2025-03-17 09:57:03.548846] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=19][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc52e38, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:3, rowkey_end_idx:-4294967296, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc52e38, micro_handle:NULL}) [2025-03-17 09:57:03.548879] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=32][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:57:03.548884] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7aaacc8a58, local_datums:0x7f7aaacc8a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:57:03.549014] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=129][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7aaacc8a58, local_datums:0x7f7aaacc8a58}, datums:[]}) [2025-03-17 09:57:03.549024] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=10][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176623505576000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:57:03.549030] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7aaacc7ac0) [2025-03-17 09:57:03.549034] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=4][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:57:03.549040] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1003},{"BIGINT":1};{"BIGINT":1003},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176653446844, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7aaacc73d0, valid:true, source:2, core:{version:{val:1742176623505576000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176623544043, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7aaacc39a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7aaacc76a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7aaacc3848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7aaacc7ac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:57:03.549324] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=283][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:03.549330] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:57:03.549334] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:57:03.549338] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=3][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:57:03.549342] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=3][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:57:03.549346] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=3][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:03.549350] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=3][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7aaacbfb50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:03.549363] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=12][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:57:03.549367] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=3][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:57:03.549377] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=9][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:57:03.549385] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=7][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:57:03.549393] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7aaacbfb50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:57:03.549402] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=9][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:03.549408] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=5][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:03.549416] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=8][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:57:03.549439] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=22][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}) [2025-03-17 09:57:03.549448] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=8][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.549472] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0077-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace890380, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222523310, assoc_session_id:3222523310, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623547246, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623544043, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222523310, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace890380}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.561957] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463550896}], scn_array=[cnt:1, {val:1742176623549366000, v:0}], curr_size=587, accum_size=2966, time_cost=12006) [2025-03-17 09:57:03.564363] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=22] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=1, size_used=65536, mem_used=126864384) [2025-03-17 09:57:03.564417] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=22] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:03.568816] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1E-0-0] [lt=30][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.568851] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1E-0-0] [lt=34][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.568893] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1E-0-0] [lt=23][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.569301] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1E-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6bec400, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222509953, assoc_session_id:3222509953, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623567521, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623567520, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222509953, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6bec400}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:03.583230] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.598780] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [117795][T1_LogLoop][T1][Y0-0000000000000000-0-0] [lt=20] LogLoopThread round_cost_time(us)(round_cost_time=12) [2025-03-17 09:57:03.602753] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708134}], scn_array=[cnt:1, {val:1742176623161152003, v:0}], curr_size=122, accum_size=1220, time_cost=40715) [2025-03-17 09:57:03.602793] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:57:03.602833] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:57:03.602896] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=1220, hit_cnt=10, read_cnt=10, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:03.604584] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=11] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:03.606687] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=31] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8fe8, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63410-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176623606158, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:491, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:03.606742] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=53][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.606774] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=31][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.606791] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=17][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.606798] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8fe8, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63410-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176623606158, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:491, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.606842] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=31][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c8fe8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63410-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176623606158, enqueue_used:2, dequeue_used:19, submit_used:1, return_used:491, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176623606832}}, part_id:-1}) [2025-03-17 09:57:03.606857] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=47][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.607174] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=26][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.607197] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=23][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.607204] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=6][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aa426315a, src_buf_size=1611) [2025-03-17 09:57:03.607210] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=5][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9cd8, ref_cnt_:1, raw_buf_:0x7f7aa4262150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63410-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176623606893, enqueue_used:1, dequeue_used:32, submit_used:1, return_used:234, callback_enqueue_used:2, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.607245] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=23][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9cd8, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63410-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176623606893, enqueue_used:1, dequeue_used:32, submit_used:1, return_used:234, callback_enqueue_used:2, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176623607235}}, part_id:-1}) [2025-03-17 09:57:03.607359] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f797408af58, id:3222493683, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf31ddc0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:03.607393] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=22] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634414} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:03.607415] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=19] end stmt(ret=0, tx_id=9634414, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf31ddc0, tx_id:{txid:9634414}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222493683, assoc_session_id:3222493683, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176623604641, active_ts:1742176623604641, commit_ts:-1, finish_ts:-1, timeout_us:29999991, lock_timeout_us:-1, expire_ts:1742176653604632, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467720220483, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623603578}], exec_info_reap_ts:1742176623603577, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623603574, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222493683, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf31ddc0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.607523] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634414}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176653604632, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623605699}{self_ls_id:{id:1}, session_id:3222493683, associated_session_id:3222493683, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634414}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467720220483, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623605699, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634414} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.607581] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=48] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634414}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176653604632, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623605699}{self_ls_id:{id:1}, session_id:3222493683, associated_session_id:3222493683, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634414}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467720220483, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623605699, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634414} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.607620] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=34] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634414}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176623607520, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:03.607635] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=13] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf31ddc0, tx_id:{txid:9634414}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222493683, assoc_session_id:3222493683, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176623604641, active_ts:1742176623604641, commit_ts:-1, finish_ts:1742176623606875, timeout_us:29999991, lock_timeout_us:-1, expire_ts:1742176653604632, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467720220483, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623603578}], exec_info_reap_ts:1742176623603577, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623603574, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:03.607700] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63410-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:03.616352] INFO [SQL.PC] update_memory_conf (ob_plan_cache.cpp:1989) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=9] update plan cache memory config(ob_plan_cache_percentage=5, ob_plan_cache_evict_high_percentage=90, ob_plan_cache_evict_low_percentage=50, tenant_id=1004) [2025-03-17 09:57:03.616378] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1491) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=23] start lib cache evict(tenant_id=1004, mem_hold=652214272, mem_limit=773094110, cache_obj_num=1799, cache_node_num=1718) [2025-03-17 09:57:03.616387] INFO [SQL.PC] cache_evict (ob_plan_cache.cpp:1508) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=8] end lib cache evict(tenant_id=1004, cache_evict_num=0, mem_hold=652214272, mem_limit=773094110, cache_obj_num=1799, cache_node_num=1718) [2025-03-17 09:57:03.616394] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2775) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=6] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:03.620030] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=24] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40424656076, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:03.619669] INFO [SQL.PC] dump_all_objs (ob_plan_cache.cpp:2493) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=4] Dumping All Cache Objs(alloc_obj_list.count()=1799, alloc_obj_list=[{obj_id:46, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179056}, {obj_id:47, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179800}, {obj_id:48, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:49, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:54, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:206, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180416}, {obj_id:207, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184298}, {obj_id:213, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:228134}, {obj_id:214, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:234205}, {obj_id:217, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207208}, {obj_id:220, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203337}, {obj_id:222, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197116}, {obj_id:223, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220151}, {obj_id:225, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208334}, {obj_id:226, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194033}, {obj_id:227, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190716}, {obj_id:228, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189404}, {obj_id:229, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204465}, {obj_id:230, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198372}, {obj_id:232, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189580}, {obj_id:233, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192620}, {obj_id:234, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189612}, {obj_id:235, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192621}, {obj_id:236, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189548}, {obj_id:237, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196031}, {obj_id:238, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191118}, {obj_id:239, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187276}, {obj_id:240, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214929}, {obj_id:241, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187052}, {obj_id:242, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192050}, {obj_id:243, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187452}, {obj_id:244, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195907}, {obj_id:245, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187148}, {obj_id:246, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188348}, {obj_id:247, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195840}, {obj_id:248, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190558}, {obj_id:258, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:260, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184937}, {obj_id:261, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:264, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190171}, {obj_id:265, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186967}, {obj_id:269, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176624}, {obj_id:271, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198826}, {obj_id:274, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:275, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189141}, {obj_id:276, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195949}, {obj_id:277, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190297}, {obj_id:278, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:279, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199077}, {obj_id:280, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178291}, {obj_id:281, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179904}, {obj_id:284, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179416}, {obj_id:285, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:217816}, {obj_id:286, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:287, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178272}, {obj_id:288, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:289, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182249}, {obj_id:290, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:292, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:293, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226000}, {obj_id:294, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:295, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:296, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:297, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202042}, {obj_id:298, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177072}, {obj_id:299, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177088}, {obj_id:305, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180218}, {obj_id:307, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189628}, {obj_id:308, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190140}, {obj_id:309, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161272}, {obj_id:310, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:311, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:312, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:169842}, {obj_id:313, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:165668}, {obj_id:314, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160523}, {obj_id:315, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:320, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:323, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188615}, {obj_id:324, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178480}, {obj_id:331, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179771}, {obj_id:333, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:341, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177952}, {obj_id:343, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180624}, {obj_id:344, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297040}, {obj_id:346, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268773}, {obj_id:349, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176557}, {obj_id:350, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178156}, {obj_id:351, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248983}, {obj_id:354, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202125}, {obj_id:355, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:359, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:252380}, {obj_id:367, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200992}, {obj_id:380, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180292}, {obj_id:385, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:386, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:393, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:400, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181377}, {obj_id:402, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181142}, {obj_id:440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182198}, {obj_id:454, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195541}, {obj_id:463, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160437}, {obj_id:464, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:475, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177152}, {obj_id:476, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:177608}, {obj_id:477, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181131}, {obj_id:499, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:510, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233226}, {obj_id:723, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198826}, {obj_id:724, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:164191}, {obj_id:859, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:875, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180552}, {obj_id:878, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194268}, {obj_id:892, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195493}, {obj_id:894, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:900, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210468}, {obj_id:902, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195461}, {obj_id:907, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202507}, {obj_id:967, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:14054}, {obj_id:968, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1234581}, {obj_id:969, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183441}, {obj_id:970, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248351}, {obj_id:973, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232386}, {obj_id:974, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209724}, {obj_id:976, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185692}, {obj_id:977, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183053}, {obj_id:981, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:279851}, {obj_id:984, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235899}, {obj_id:985, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:986, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:235514}, {obj_id:993, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236395}, {obj_id:997, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:209553}, {obj_id:998, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:384189}, {obj_id:1003, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:346943}, {obj_id:1004, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1036860}, {obj_id:1005, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214841}, {obj_id:1013, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180726}, {obj_id:1017, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179887}, {obj_id:1021, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197096}, {obj_id:1022, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322960}, {obj_id:1025, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200656}, {obj_id:1026, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340153}, {obj_id:1027, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:322500}, {obj_id:1031, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:434216}, {obj_id:1033, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:328057}, {obj_id:1036, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206700}, {obj_id:1037, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192229}, {obj_id:1038, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:283899}, {obj_id:1048, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1049, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1053, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163056}, {obj_id:1061, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203111}, {obj_id:1065, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:242505}, {obj_id:1066, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207162}, {obj_id:1067, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195590}, {obj_id:1069, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187300}, {obj_id:1070, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185320}, {obj_id:1074, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176744}, {obj_id:1077, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:187396}, {obj_id:1079, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:1080, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:1081, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1082, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:1083, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178440}, {obj_id:1084, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183537}, {obj_id:1085, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181251}, {obj_id:1088, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1090, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191548}, {obj_id:1092, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:453315}, {obj_id:1105, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264567}, {obj_id:1111, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196613}, {obj_id:1115, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196661}, {obj_id:1117, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1218, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1221, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1223, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1226, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1229, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1232, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1235, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1238, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1242, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1245, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1247, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1251, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1254, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1256, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1260, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1263, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1265, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1269, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1272, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1274, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1278, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1281, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1283, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1334, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204728}, {obj_id:1337, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1340, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1342, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1344, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1346, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1348, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1350, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1352, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1355, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1358, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1360, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1363, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1366, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1368, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1371, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1374, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1376, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1379, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1382, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1384, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1387, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1390, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:1392, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:1463, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196277}, {obj_id:1466, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196325}, {obj_id:1552, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197045}, {obj_id:1555, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197093}, {obj_id:1761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183538}, {obj_id:1762, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182131}, {obj_id:1764, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182340}, {obj_id:1767, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:1768, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182686}, {obj_id:1769, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182290}, {obj_id:1770, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184363}, {obj_id:1771, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180844}, {obj_id:1772, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181059}, {obj_id:1773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184817}, {obj_id:1774, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183668}, {obj_id:1775, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184103}, {obj_id:1776, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183797}, {obj_id:1786, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:198904}, {obj_id:1787, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331052}, {obj_id:1790, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:202496}, {obj_id:1791, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:348722}, {obj_id:1792, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:331352}, {obj_id:1799, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178907}, {obj_id:1800, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179715}, {obj_id:1801, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181537}, {obj_id:1802, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182612}, {obj_id:1803, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260440}, {obj_id:1804, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:230691}, {obj_id:1806, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:246344}, {obj_id:1808, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223882}, {obj_id:1809, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:448756}, {obj_id:1818, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188140}, {obj_id:1820, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191323}, {obj_id:1823, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1825, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:244350}, {obj_id:1871, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:250368}, {obj_id:1872, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207905}, {obj_id:1873, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180710}, {obj_id:1874, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184839}, {obj_id:1875, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182239}, {obj_id:1880, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:268450}, {obj_id:1881, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:214951}, {obj_id:1882, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:659525}, {obj_id:1887, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272054}, {obj_id:1888, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:232022}, {obj_id:1889, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:676458}, {obj_id:1894, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206106}, {obj_id:1895, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:255666}, {obj_id:1896, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:364093}, {obj_id:1897, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:349058}, {obj_id:1901, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:248939}, {obj_id:1905, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:220341}, {obj_id:1906, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208181}, {obj_id:1907, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:431472}, {obj_id:1908, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:419831}, {obj_id:1912, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:222082}, {obj_id:1913, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:439892}, {obj_id:1918, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218531}, {obj_id:1919, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:422873}, {obj_id:1920, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:410978}, {obj_id:1924, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185511}, {obj_id:1931, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:336934}, {obj_id:1932, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:295855}, {obj_id:1933, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:1947, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204286}, {obj_id:1949, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:356276}, {obj_id:1950, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:340211}, {obj_id:1959, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211545}, {obj_id:1964, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:301536}, {obj_id:1965, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:260857}, {obj_id:1975, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:201395}, {obj_id:1987, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184389}, {obj_id:2008, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239940}, {obj_id:2010, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:525534}, {obj_id:2016, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:218641}, {obj_id:2022, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191835}, {obj_id:2023, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:297162}, {obj_id:2024, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:278315}, {obj_id:2035, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:207891}, {obj_id:2036, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:375309}, {obj_id:2037, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:357951}, {obj_id:2058, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:193588}, {obj_id:2059, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:305064}, {obj_id:2060, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:287152}, {obj_id:2081, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184971}, {obj_id:2109, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:216802}, {obj_id:2110, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:413810}, {obj_id:2111, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:402141}, {obj_id:2129, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225681}, {obj_id:2130, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:459054}, {obj_id:2131, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:446413}, {obj_id:2171, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184227}, {obj_id:2289, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184631}, {obj_id:2376, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:204811}, {obj_id:2395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221640}, {obj_id:2402, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:239174}, {obj_id:2431, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:4, added_to_lc:true, mem_used:178400}, {obj_id:2435, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178400}, {obj_id:2440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184354}, {obj_id:2441, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182588}, {obj_id:2442, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210883}, {obj_id:2444, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210950}, {obj_id:2446, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197694}, {obj_id:2448, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179737}, {obj_id:2452, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:233751}, {obj_id:2502, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188484}, {obj_id:2507, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179273}, {obj_id:2573, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185063}, {obj_id:2574, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196061}, {obj_id:2589, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180547}, {obj_id:2590, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180525}, {obj_id:2593, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176624}, {obj_id:2632, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:213894}, {obj_id:2634, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180532}, {obj_id:2635, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:180529}, {obj_id:2651, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186780}, {obj_id:2652, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181185}, {obj_id:2653, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182472}, {obj_id:2654, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:178176}, {obj_id:2655, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181137}, {obj_id:2656, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:181817}, {obj_id:2677, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195622}, {obj_id:2678, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2679, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2680, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2681, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160496}, {obj_id:2682, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161287}, {obj_id:2686, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208041}, {obj_id:2688, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184389}, {obj_id:2690, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191552}, {obj_id:2693, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2714, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:236571}, {obj_id:2743, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:163054}, {obj_id:2761, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:183342}, {obj_id:2765, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223598}, {obj_id:2773, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223592}, {obj_id:2814, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:298681}, {obj_id:2827, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2835, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2836, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:2837, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:179776}, {obj_id:2838, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185454}, {obj_id:2952, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:211463}, {obj_id:2953, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203185}, {obj_id:2954, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:274521}, {obj_id:3013, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:272510}, {obj_id:3108, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224378}, {obj_id:3165, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:224762}, {obj_id:3186, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192084}, {obj_id:3187, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:203098}, {obj_id:3231, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225178}, {obj_id:3390, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:206653}, {obj_id:3391, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:186851}, {obj_id:3392, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190051}, {obj_id:3394, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:192545}, {obj_id:3395, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190052}, {obj_id:3396, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:184534}, {obj_id:3397, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190152}, {obj_id:3408, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3409, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:208648}, {obj_id:3412, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:221865}, {obj_id:3413, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:226011}, {obj_id:3414, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185340}, {obj_id:3415, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191637}, {obj_id:3416, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:342020}, {obj_id:3417, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:428864}, {obj_id:3428, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:185714}, {obj_id:3429, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:188924}, {obj_id:3436, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:194201}, {obj_id:3437, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:245052}, {obj_id:3438, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296079}, {obj_id:3439, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:225978}, {obj_id:3440, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196807}, {obj_id:3441, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197829}, {obj_id:3442, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:296075}, {obj_id:3443, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:191830}, {obj_id:3444, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:197877}, {obj_id:3445, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189148}, {obj_id:3446, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3447, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3456, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3457, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3466, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3467, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3470, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3472, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3481, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:182711}, {obj_id:3482, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:189007}, {obj_id:3483, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3485, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3488, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3490, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3495, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3501, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3527, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:190940}, {obj_id:3531, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:210225}, {obj_id:3538, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:196295}, {obj_id:3560, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:264475}, {obj_id:3635, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:199341}, {obj_id:3644, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:195619}, {obj_id:3645, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:160477}, {obj_id:3646, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:161284}, {obj_id:3656, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3657, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3661, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3685, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:3693, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3696, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3699, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3702, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3705, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3708, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3711, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3713, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176714}, {obj_id:3730, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:1752}, {obj_id:3731, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:258706}, {obj_id:3732, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:200264}, {obj_id:3735, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3738, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3741, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3744, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3747, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:176788}, {obj_id:3749, tenant_id:1004, log_del_time:9223372036854775807, real_del_time:9223372036854775807, ref_count:2, added_to_lc:true, mem_used:223626}, { [2025-03-17 09:57:03.621101] INFO [SQL.PC] runTimerTask (ob_plan_cache.cpp:2783) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=1429] schedule next cache evict task(evict_interval=5000000) [2025-03-17 09:57:03.630687] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=1, size_used=0, mem_used=22877184) [2025-03-17 09:57:03.630759] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=13] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:03.635333] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.635436] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.635448] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=11] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.635716] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=32] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636120] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=23] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636383] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636395] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=16] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636439] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=18] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636510] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=22] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.636521] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=19] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:03.641790] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=29] gc stale ls task succ [2025-03-17 09:57:03.644898] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=8] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:723, free_obj_hold_cnt:78, allocator used:3082248, allocator total:3329280}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:233, free_obj_hold_cnt:724, allocator used:62770587, allocator total:62770587}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():130944, full allocator used:1600, full allocator total:196480}, tablets_mem=66296347, tablets_mem_limit=3092376440, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:25, free_obj_hold_cnt:88, allocator used:231424, allocator total:261120}, pending_cnt=0, wait_gc_count=0, tablet count=956) [2025-03-17 09:57:03.649703] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=52] new mysql sessid created(conn.sessid_=3222522688, support_ssl=false) [2025-03-17 09:57:03.649750] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=24] sm conn init succ(conn.sessid_=3222522688, sess.client_addr_="192.168.1.62:36396") [2025-03-17 09:57:03.649760] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=8] accept one succ(*s={this:0x7f7af9aa4db0, session_id:3222522688, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:300}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.649824] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117624][sql_nio5][T1][Y0-0000000000000000-0-0] [lt=15] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:03.649857] INFO [SERVER] check_common_property (obmp_connect.cpp:2034) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A2-0-0] [lt=24] construct session id(conn.client_sessid_=268621, conn.sessid_=3222522688, conn.client_addr_port_=36396, conn.client_create_time_=1741767280926750, conn.proxy_sessid_=723682327204659507, ObLocalDiagnosticInfo::get()={tenant_id:1, group_id:0, session_id:3222522688, curr_wait:event_no:0, p1:0, p2:0, p3:0, wait_begin_time:0, wait_end_time:0, wait_time:0, timeout_ms:0, level:0, parent:0, event_id:10000, event_name:"", param1:"", param2:"", param3:"", ref_cnt:2, need_aggregate:true, get_uref():1073741824, get_href():2, ash_stat:{ObActiveSessionStatItem:{tenant_id:1, session_id:3222522688, event id:10000, event:"", wait_time:0, time_model:0, trace_id:Y0-00062FB7EF5684A2-0-0, plan_line_id:-1, sql_id:"", top_level_sql_id:"", plsql_entry_subprogram_name:"", plsql_subprogram_name:"", session_type:0, is_wr_sample:false, delta_time:0, delta_cpu_time:0, delta_db_time:0, program:"T1_SQL_CMD", module:"", action:"", client_id:"", group_id:0, tid:117813, plan_hash:0, tx_id:0, stmt_type:0, tablet_id:0, block_sessid:0}, last_touch_ts:1894405500344942, wait_event_begin_ts:0, total_idle_wait_time:0, total_non_idle_wait_time:0, prev_idle_wait_time:0, prev_non_idle_wait_time:0, total_cpu_time:0, is_active_session:true, inner_sql_wait_type_id:0, pcode:0, tm_idle_time:29, fixup_index:-1, fixup_ash_buffer:{this:0x7f7af2610af8, block_ptr_.control_ptr:null, block_ptr_.data_ptr:null}, retry_wait_event_no:0, retry_plan_line_id:-1}}) [2025-03-17 09:57:03.650955] WDIAG [RPC.OBMYSQL] serialize (ompk_ok.cpp:262) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A2-0-0] [lt=68][errcode=-4019] size is overflow(length=13312, pos=4, need_size=17065, ret=-4019) [2025-03-17 09:57:03.651033] INFO [SERVER] process (obmp_connect.cpp:523) [117813][T1_MysqlQueueTh][T1][Y0-00062FB7EF5684A2-0-0] [lt=27] MySQL LOGIN(direct_client_ip="192.168.1.62", client_ip=192.168.1.62, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3222522688, proxy_sessid=723682327204659507, client_sessid=268621, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=12172103, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:03.651307] INFO [SERVER] process (obmp_init_db.cpp:53) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB7F06F5722-0-0] [lt=16] init db(db_name=oceanbase) [2025-03-17 09:57:03.651962] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=50][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=76, len=440, pos=423) [2025-03-17 09:57:03.652532] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=23][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=49, len=257, pos=227) [2025-03-17 09:57:03.653194] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=8, len=106, pos=83) [2025-03-17 09:57:03.653852] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=6, len=90, pos=66) [2025-03-17 09:57:03.654348] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=43] ====== check clog disk timer task ====== [2025-03-17 09:57:03.654373] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=21] get_disk_usage(ret=0, capacity(MB):=4915, used(MB):=3930) [2025-03-17 09:57:03.654548] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=74, len=314, pos=284) [2025-03-17 09:57:03.655199] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=14][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=23, len=208, pos=179) [2025-03-17 09:57:03.655804] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5723-0-0] [lt=11][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=76, len=445, pos=423) [2025-03-17 09:57:03.656163] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=6] new mysql sessid created(conn.sessid_=3222523431, support_ssl=false) [2025-03-17 09:57:03.656204] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=18] sm conn init succ(conn.sessid_=3222523431, sess.client_addr_="192.168.1.62:36398") [2025-03-17 09:57:03.656220] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=13] accept one succ(*s={this:0x7f7af9aa8db0, session_id:3222523431, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:301}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.656292] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117622][sql_nio3][T1][Y0-0000000000000000-0-0] [lt=21] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:03.656321] INFO [SERVER] check_common_property (obmp_connect.cpp:2034) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED68361-0-0] [lt=37] construct session id(conn.client_sessid_=268621, conn.sessid_=3222523431, conn.client_addr_port_=36398, conn.client_create_time_=1741767280926750, conn.proxy_sessid_=723682327204659507, ObLocalDiagnosticInfo::get()={tenant_id:1, group_id:0, session_id:3222523431, curr_wait:event_no:0, p1:0, p2:0, p3:0, wait_begin_time:0, wait_end_time:0, wait_time:0, timeout_ms:0, level:0, parent:0, event_id:10000, event_name:"", param1:"", param2:"", param3:"", ref_cnt:2, need_aggregate:true, get_uref():1073741824, get_href():2, ash_stat:{ObActiveSessionStatItem:{tenant_id:1, session_id:3222523431, event id:10000, event:"", wait_time:0, time_model:0, trace_id:Y0-00062FB7EED68361-0-0, plan_line_id:-1, sql_id:"", top_level_sql_id:"", plsql_entry_subprogram_name:"", plsql_subprogram_name:"", session_type:0, is_wr_sample:false, delta_time:0, delta_cpu_time:0, delta_db_time:0, program:"T1_SQL_CMD", module:"", action:"", client_id:"", group_id:0, tid:117812, plan_hash:0, tx_id:0, stmt_type:0, tablet_id:0, block_sessid:0}, last_touch_ts:1894405514539886, wait_event_begin_ts:0, total_idle_wait_time:0, total_non_idle_wait_time:0, prev_idle_wait_time:0, prev_non_idle_wait_time:0, total_cpu_time:0, is_active_session:true, inner_sql_wait_type_id:0, pcode:0, tm_idle_time:27, fixup_index:-1, fixup_ash_buffer:{this:0x7f7a603e0af8, block_ptr_.control_ptr:null, block_ptr_.data_ptr:null}, retry_wait_event_no:0, retry_plan_line_id:-1}}) [2025-03-17 09:57:03.657109] WDIAG [RPC.OBMYSQL] serialize (ompk_ok.cpp:262) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED68361-0-0] [lt=56][errcode=-4019] size is overflow(length=13312, pos=4, need_size=17065, ret=-4019) [2025-03-17 09:57:03.657182] INFO [SERVER] process (obmp_connect.cpp:523) [117812][T1_MysqlQueueTh][T1][Y0-00062FB7EED68361-0-0] [lt=18] MySQL LOGIN(direct_client_ip="192.168.1.62", client_ip=192.168.1.62, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3222523431, proxy_sessid=723682327204659507, client_sessid=268621, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=12172103, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:03.657389] INFO [SERVER] process (obmp_init_db.cpp:53) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB937B85906-0-0] [lt=23] init db(db_name=oceanbase) [2025-03-17 09:57:03.661204] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB937B85909-0-0] [lt=19][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:03.662085] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:03.662114] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:03.662137] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:03.662161] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:03.664310] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FBC5FE2A-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.664372] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539400][T1003_DupTbLeas][T1][YB420A0B093E-00062FB7FBC5FE2A-0-0] [lt=17][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:03.665289] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539400][T1003_DupTbLeas][T1][YB420A0B093E-00062FB7FBC5FE2A-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ad8713be0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222514706, assoc_session_id:3222514706, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623663449, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623661499, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222514706, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ad8713be0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.665446] WDIAG [STORAGE.DUP_TABLE] execute_for_dup_ls_ (ob_dup_table_util.cpp:207) [3539400][T1003_DupTbLeas][T1003][Y0-0000000000000000-0-0] [lt=1][errcode=0] refresh dup ls failed(tmp_ret=-4016, this={tenant_id_:1003, dup_table_scan_timer_:0x7f79a543dae8, dup_loop_worker_:0x7f79a57066f0, min_dup_ls_status_info_:{tenant_id:0, ls_id:{id:-1}, ls_group_id:18446744073709551615, status:"UNKNOWN", unit_group_id:18446744073709551615, primary_zone:"", flag:{flag:0, is_duplicate:false, is_block_tablet_in:false}}, tenant_schema_dup_tablet_set_.size():0, scan_task_execute_interval_:10000000, last_dup_ls_refresh_time_:0, last_dup_schema_refresh_time_:0, last_scan_task_succ_time_:1742176611657938, max_execute_interval_:12970703}) [2025-03-17 09:57:03.672280] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=22] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=2) [2025-03-17 09:57:03.674276] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=21] gc stale ls task succ [2025-03-17 09:57:03.682103] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421812}], scn_array=[cnt:1, {val:1742176623161152004, v:0}], curr_size=122, accum_size=366, time_cost=19938) [2025-03-17 09:57:03.682324] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=12][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708256}], scn_array=[cnt:1, {val:1742176623161152004, v:0}], curr_size=122, accum_size=122, time_cost=20231) [2025-03-17 09:57:03.683300] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.687998] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=31][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=9, len=122, pos=91) [2025-03-17 09:57:03.688532] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=19][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=481, pos=299) [2025-03-17 09:57:03.689015] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=15, pos=15) [2025-03-17 09:57:03.689494] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=60, len=299, pos=269) [2025-03-17 09:57:03.689932] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=25, len=233, pos=204) [2025-03-17 09:57:03.690398] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=23, pos=15) [2025-03-17 09:57:03.690839] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118847][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5727-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=74, pos=48) [2025-03-17 09:57:03.704063] INFO [RPC.OBMYSQL] sm_conn_build_handshake (obsm_conn_callback.cpp:122) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=11] new mysql sessid created(conn.sessid_=3222512600, support_ssl=false) [2025-03-17 09:57:03.704115] INFO [RPC.OBMYSQL] init (obsm_conn_callback.cpp:142) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=21] sm conn init succ(conn.sessid_=3222512600, sess.client_addr_="192.168.1.62:36404") [2025-03-17 09:57:03.704137] INFO [RPC.OBMYSQL] do_accept_one (ob_sql_nio.cpp:1103) [117620][sql_nio1][T0][Y0-0000000000000000-0-0] [lt=17] accept one succ(*s={this:0x7f7af9b46db0, session_id:3222512600, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:-1, sql_initiative_shutdown:false, reader:{fd:304}, err:0, last_decode_time:0, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:0, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.704219] INFO [SERVER] dispatch_req (ob_srv_deliver.cpp:287) [117620][sql_nio1][T1][Y0-0000000000000000-0-0] [lt=36] succeed to dispatch to tenant mysql queue(tenant_id=1) [2025-03-17 09:57:03.704249] INFO [SERVER] check_common_property (obmp_connect.cpp:2034) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F056849A-0-0] [lt=25] construct session id(conn.client_sessid_=268621, conn.sessid_=3222512600, conn.client_addr_port_=36404, conn.client_create_time_=1741767280926750, conn.proxy_sessid_=723682327204659507, ObLocalDiagnosticInfo::get()={tenant_id:1, group_id:0, session_id:3222512600, curr_wait:event_no:0, p1:0, p2:0, p3:0, wait_begin_time:0, wait_end_time:0, wait_time:0, timeout_ms:0, level:0, parent:0, event_id:10000, event_name:"", param1:"", param2:"", param3:"", ref_cnt:2, need_aggregate:true, get_uref():1073741824, get_href():2, ash_stat:{ObActiveSessionStatItem:{tenant_id:1, session_id:3222512600, event id:10000, event:"", wait_time:0, time_model:0, trace_id:Y0-00062FB7F056849A-0-0, plan_line_id:-1, sql_id:"", top_level_sql_id:"", plsql_entry_subprogram_name:"", plsql_subprogram_name:"", session_type:0, is_wr_sample:false, delta_time:0, delta_cpu_time:0, delta_db_time:0, program:"T1_SQL_CMD", module:"", action:"", client_id:"", group_id:0, tid:117810, plan_hash:0, tx_id:0, stmt_type:0, tablet_id:0, block_sessid:0}, last_touch_ts:1894405619729546, wait_event_begin_ts:0, total_idle_wait_time:0, total_non_idle_wait_time:0, prev_idle_wait_time:0, prev_non_idle_wait_time:0, total_cpu_time:0, is_active_session:true, inner_sql_wait_type_id:0, pcode:0, tm_idle_time:28, fixup_index:-1, fixup_ash_buffer:{this:0x7f7af2611f98, block_ptr_.control_ptr:null, block_ptr_.data_ptr:null}, retry_wait_event_no:0, retry_plan_line_id:-1}}) [2025-03-17 09:57:03.705262] WDIAG [RPC.OBMYSQL] serialize (ompk_ok.cpp:262) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F056849A-0-0] [lt=68][errcode=-4019] size is overflow(length=13312, pos=4, need_size=17065, ret=-4019) [2025-03-17 09:57:03.705317] INFO [SERVER] process (obmp_connect.cpp:523) [117810][T1_MysqlQueueTh][T1][Y0-00062FB7F056849A-0-0] [lt=20] MySQL LOGIN(direct_client_ip="192.168.1.62", client_ip=192.168.1.62, tenant_name=sys, tenant_id=1, user_name=proxyro, host_name=%, sessid=3222512600, proxy_sessid=723682327204659507, client_sessid=268621, from_proxy=true, from_java_client=false, from_oci_client=false, from_jdbc_client=false, capability=150972045, proxy_capability=12172103, use_ssl=false, c/s protocol="OB_2_0_CS_TYPE", autocommit=true, proc_ret=0, ret=0, conn->client_type_=3, conn->client_version_=0) [2025-03-17 09:57:03.705563] INFO [SERVER] process (obmp_init_db.cpp:53) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB8C01ACB02-0-0] [lt=25] init db(db_name=oceanbase) [2025-03-17 09:57:03.707474] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117887][T1_L0_G0][T1][YB420A0B093E-00062FB8C01ACB04-0-0] [lt=18][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:03.710933] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=14][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9aa8db0, session_id:3222523431, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:301}, err:5, last_decode_time:1742176623658160, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1101, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.710976] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=36] kill and revert session(conn.sessid_=3222523431, proxy_sessid=723682327204659507, ret=0) [2025-03-17 09:57:03.710946] WDIAG [RPC.OBMYSQL] handle_sock_event (ob_sql_nio.cpp:1022) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=6][errcode=0] socket closed, it maybe disconnected by the client or by observer actively(mask=8197, *s={this:0x7f7af9aa4db0, session_id:3222522688, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:300}, err:5, last_decode_time:1742176623651765, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:752, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.710985] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:997) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=8] sock ref clean, do destroy(*s={this:0x7f7af9aa8db0, session_id:3222523431, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:301}, err:5, last_decode_time:1742176623658160, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:1101, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.710991] INFO [RPC.OBMYSQL] on_disconnect (obsm_conn_callback.cpp:272) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=39] kill and revert session(conn.sessid_=3222522688, proxy_sessid=723682327204659507, ret=0) [2025-03-17 09:57:03.711002] INFO [RPC.OBMYSQL] handle_pending_destroy_list (ob_sql_nio.cpp:997) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=10] sock ref clean, do destroy(*s={this:0x7f7af9aa4db0, session_id:3222522688, trace_id:Y0-0000000000000000-0-0, sql_handling_stage:256, sql_initiative_shutdown:false, reader:{fd:300}, err:5, last_decode_time:1742176623651765, pending_write_task:{buf:null, sz:0}, need_epoll_trigger_write:false, consume_size:752, pending_flag:0, may_handling_flag:true, handler_close_flag:false}) [2025-03-17 09:57:03.711015] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=10] unlock session of tenant(conn.sessid_=3222523431, proxy_sessid=723682327204659507, conn.tenant_id_=1) [2025-03-17 09:57:03.711039] INFO [RPC.OBMYSQL] sm_conn_unlock_tenant (obsm_conn_callback.cpp:160) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=14] unlock session of tenant(conn.sessid_=3222522688, proxy_sessid=723682327204659507, conn.tenant_id_=1) [2025-03-17 09:57:03.711062] INFO [SERVER] run (obmp_disconnect.cpp:83) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=31] free session successfully(sessid=3222523431) [2025-03-17 09:57:03.711081] INFO [SERVER] run (obmp_disconnect.cpp:90) [117887][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=19] mark session id unused(sessid=3222523431) [2025-03-17 09:57:03.711023] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117622][sql_nio3][T0][Y0-0000000000000000-0-0] [lt=7] connection close(sessid=3222523431, proxy_sessid=723682327204659507, tenant_id=1, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB937B85909-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:11, cur_pkt_pos_:17, pkt_rec[0]:{send:obp_mysql_header_{len_:17069, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp20_header_:{payload_len:14, pkt_seq:0, req_id:6018757, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:646, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[2]:{send:obp20_header_:{payload_len:43, pkt_seq:0, req_id:6018758, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:621, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{receive:obp20_header_:{payload_len:282, pkt_seq:0, req_id:6018759, flag_.flags:26, ext_flags:0}, obp_mysql_header_{len_:278, rec_:282, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[4]:{send:obp_compess_header_{com_len:668, com_seq:1}, obp20_header_:{payload_len:640, pkt_seq:1, req_id:6018759, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_compess_header_{com_len:668, com_seq:1}, obp20_header_:{payload_len:640, pkt_seq:1, req_id:6018759, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:7, seq_:9}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_compess_header_{com_len:668, com_seq:1}, obp20_header_:{payload_len:640, pkt_seq:1, req_id:6018759, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:9, seq_:10}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{send:obp_compess_header_{com_len:668, com_seq:1}, obp20_header_:{payload_len:640, pkt_seq:1, req_id:6018759, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:4, seq_:14}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[8]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:6018759, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:35, seq_:15}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:6018759, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:12, seq_:16}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[10]:{receive:obp20_header_:{payload_len:194, pkt_seq:0, req_id:6018760, flag_.flags:30, ext_flags:0}, obp_mysql_header_{len_:190, rec_:194, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[11]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[12]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[13]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[14]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:15}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[15]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:9, seq_:16}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[16]:{send:obp_compess_header_{com_len:836, com_seq:1}, obp20_header_:{payload_len:808, pkt_seq:1, req_id:6018760, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:12, seq_:17}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2025-03-17 09:57:03.711082] INFO [SERVER] run (obmp_disconnect.cpp:83) [117888][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=24] free session successfully(sessid=3222522688) [2025-03-17 09:57:03.711096] INFO [SERVER] run (obmp_disconnect.cpp:90) [117888][T1_L0_G0][T1][Y0-0000000000000000-0-0] [lt=13] mark session id unused(sessid=3222522688) [2025-03-17 09:57:03.711053] INFO [RPC.OBMYSQL] destroy (obsm_conn_callback.cpp:248) [117624][sql_nio5][T0][Y0-0000000000000000-0-0] [lt=14] connection close(sessid=3222522688, proxy_sessid=723682327204659507, tenant_id=1, from_proxy=true, from_java_client=false, c/s protocol="OB_2_0_CS_TYPE", is_need_clear_sessid_=true, is_sess_alloc_=true, ret=0, trace_id=YB420A0B093E-00062FB7F06F5725-0-0, conn.pkt_rec_wrapper_=[start_pkt_pos_:8, cur_pkt_pos_:10, pkt_rec[0]:{send:obp_mysql_header_{len_:17069, seq_:3}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[1]:{send:obp20_header_:{payload_len:14, pkt_seq:0, req_id:9484120, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:646, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[2]:{send:obp20_header_:{payload_len:43, pkt_seq:0, req_id:9484121, flag_.flags:10, ext_flags:0}, obp_mysql_header_{len_:621, seq_:2}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}, pkt_rec[3]:{receive:obp20_header_:{payload_len:162, pkt_seq:0, req_id:9484122, flag_.flags:26, ext_flags:0}, obp_mysql_header_{len_:158, rec_:162, seq_:0}, pkt_name:"COM_QUERY", obp_mysql_header_.type_:3, obp_mysql_header_.is_send_:0}, pkt_rec[4]:{send:obp_compess_header_{com_len:476, com_seq:1}, obp20_header_:{payload_len:448, pkt_seq:1, req_id:9484122, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:5, seq_:2}, pkt_name:"PKT_RESHEAD", obp_mysql_header_.is_send_:1}, pkt_rec[5]:{send:obp_compess_header_{com_len:476, com_seq:1}, obp20_header_:{payload_len:448, pkt_seq:1, req_id:9484122, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:6, seq_:8}, pkt_name:"PKT_FIELD", obp_mysql_header_.is_send_:1}, pkt_rec[6]:{send:obp_compess_header_{com_len:476, com_seq:1}, obp20_header_:{payload_len:448, pkt_seq:1, req_id:9484122, flag_.flags:8, ext_flags:0}, obp_mysql_header_{len_:9, seq_:9}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[7]:{send:obp_compess_header_{com_len:476, com_seq:1}, obp20_header_:{payload_len:448, pkt_seq:1, req_id:9484122, flag_.flags:8, ext_flags:0}, obp_mysql_header_{pkt_num_:1, seq_:10}, pkt_name:"PKT_ROW", obp_mysql_header_.is_send_:1}, pkt_rec[8]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:9484122, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:35, seq_:11}, pkt_name:"PKT_EOF", obp_mysql_header_.is_send_:1}, pkt_rec[9]:{send:obp_compess_header_{com_len:75, com_seq:2}, obp20_header_:{payload_len:47, pkt_seq:2, req_id:9484122, flag_.flags:11, ext_flags:0}, obp_mysql_header_{len_:12, seq_:12}, pkt_name:"PKT_OKP", obp_mysql_header_.is_send_:1}], disconnect_state=4) [2025-03-17 09:57:03.715451] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:03.716521] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=19] Compaction Mem Pool current stat: (ret=0, max_block_num=64, total_block_num=16, Chunk list size: =2) [2025-03-17 09:57:03.717622] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.717656] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=33][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.717685] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=28][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:03.717693] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6648, ref_cnt_:1, raw_buf_:0x7f791d1c2150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63411-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176623717133, enqueue_used:2, dequeue_used:25, submit_used:2, return_used:443, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.717726] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=28][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6648, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63411-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176623717133, enqueue_used:2, dequeue_used:25, submit_used:2, return_used:443, callback_enqueue_used:3, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176623717724}}, part_id:-1}) [2025-03-17 09:57:03.718012] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=19][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:03.718044] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=31][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:03.718071] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=26][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79f029915a, src_buf_size=1611) [2025-03-17 09:57:03.718081] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c94b0, ref_cnt_:1, raw_buf_:0x7f79f0298150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63411-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a2293c080, time_log:{begin_ts:1742176623717741, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:243, callback_enqueue_used:1, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:03.718123] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=36][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c94b0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63411-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:null, time_log:{begin_ts:1742176623717741, enqueue_used:1, dequeue_used:15, submit_used:1, return_used:243, callback_enqueue_used:1, callback_dequeue_used:8, callback_finish_used:-1, end_used:1742176623718119}}, part_id:-1}) [2025-03-17 09:57:03.718221] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974082038, id:3222516614, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3e01c0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:03.718258] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=25] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:6}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634416} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:03.718283] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=21] end stmt(ret=0, tx_id=9634416, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3e01c0, tx_id:{txid:9634416}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516614, assoc_session_id:3222516614, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176623715644, active_ts:1742176623715644, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653715638, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467831303374, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623714565}], exec_info_reap_ts:1742176623714564, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623714561, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974082038, id:3222516614, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3e01c0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:03.718385] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634416}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176653715638, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623716717}{self_ls_id:{id:1}, session_id:3222516614, associated_session_id:3222516614, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634416}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467831303374, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623716717, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634416} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.718454] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=59] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634416}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176653715638, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176623716717}{self_ls_id:{id:1}, session_id:3222516614, associated_session_id:3222516614, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634416}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409467831303374, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176623716717, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634416} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:03.718494] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=36] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634416}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176623718381, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:03.718510] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3e01c0, tx_id:{txid:9634416}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222516614, assoc_session_id:3222516614, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176623715644, active_ts:1742176623715644, commit_ts:-1, finish_ts:1742176623717786, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176653715638, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409467831303374, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176623714565}], exec_info_reap_ts:1742176623714564, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176623714561, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:03.718577] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63411-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:03.741415] INFO [COORDINATOR] refresh (ob_leader_coordinator.cpp:146) [3539354][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=23] refresh __all_ls_election_reference_info success(ret=0, ret="OB_SUCCESS", *new_all_ls_election_reference_info=[{1,0,False,{False,},False,False,True}]) [2025-03-17 09:57:03.750743] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859730020}], scn_array=[cnt:1, {val:1742176623734927000, v:0}], curr_size=587, accum_size=1175, time_cost=15572) [2025-03-17 09:57:03.750943] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=31] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:57:03.769304] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [117834][T1_DBMSSched][T1][YB420A0B093E-00062FB7F38CDC93-0-0] [lt=29] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:57:03.772784] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=42][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463551605}], scn_array=[cnt:1, {val:1742176623748781000, v:0}], curr_size=587, accum_size=3675, time_cost=23108) [2025-03-17 09:57:03.772820] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162421934}], scn_array=[cnt:1, {val:1742176623662393000, v:0}], curr_size=122, accum_size=488, time_cost=10575) [2025-03-17 09:57:03.772851] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708378}], scn_array=[cnt:1, {val:1742176623662393000, v:0}], curr_size=122, accum_size=244, time_cost=10643) [2025-03-17 09:57:03.773050] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=35] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1, new_min_version={val:1742176623549366000, v:0}, new_max_version={val:1742176623549366000, v:0}, last_min_version={val:1742176623348986000, v:0}, last_max_version={val:1742176623348986000, v:0}, total_time_us=24568, delta_us=223682) [2025-03-17 09:57:03.773232] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=178] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1, current_version={val:1742176623549366000, v:0}, min_version={val:1742176623549366000, v:0}, max_version={val:1742176623549366000, v:0}, in_service=true, leader_epoch=4) [2025-03-17 09:57:03.779035] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=35] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:03.783332] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=25][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.791189] INFO [STORAGE] try_shrink (ob_compaction_memory_pool.cpp:522) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=19] Compaction Mem Pool current stat: (ret=0, max_block_num=64, total_block_num=16, Chunk list size: =2) [2025-03-17 09:57:03.820386] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=39] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40420461772, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:03.824585] INFO [LIB] runTimerTask (ob_work_queue.cpp:24) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=29] add async task(this=task_type:N9oceanbase10rootserver21ObPurgeRecyclebinTaskE, timeout_check:False) [2025-03-17 09:57:03.862304] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=19] QSched: |cfg: root:1,x,0 net_in:1,1203M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:280K/66:15830 disk:280K/66:15830 t1m2:100K/23:15883 t1m2g0:100K/23:15883 t1004m2:92K/23:15619 t1004m2g0:92K/23:15619 t1003m2:88K/20:16010 t1003m2g0:88K/20:16010 |q0: 0 0 0 |q1: 0 0 0 [2025-03-17 09:57:03.872378] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=11] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=1) [2025-03-17 09:57:03.875617] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422056}], scn_array=[cnt:1, {val:1742176623662393001, v:0}], curr_size=122, accum_size=610, time_cost=13383) [2025-03-17 09:57:03.880974] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=12] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=4506) [2025-03-17 09:57:03.881566] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708500}], scn_array=[cnt:1, {val:1742176623662393001, v:0}], curr_size=122, accum_size=366, time_cost=19341) [2025-03-17 09:57:03.883413] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.898382] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539427][T1003_TenantWea][T1003][YB420A0B093E-00062FB95B26709A-0-0] [lt=24] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.193825001716613770e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=596, Throttle Trigger(MB)=358, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.394390525804035121e+02) [2025-03-17 09:57:03.933078] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43] [PALF STAT WRITE LOG COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=19713, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:57:03.933112] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=32][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859730607}], scn_array=[cnt:1, {val:1742176623897539000, v:0}], curr_size=587, accum_size=1762, time_cost=34438) [2025-03-17 09:57:03.933206] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8347) [2025-03-17 09:57:03.940834] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=10] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312809281801, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:03.940870] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=33][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:57:03.940882] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=11] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7aeb5bf7b0) [2025-03-17 09:57:03.940889] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=6] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312809281801, tenant_id:500}, cur_thread_count=3, cnt=-1, new_thread_count=2) [2025-03-17 09:57:03.940954] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=23][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:57:03.940983] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:57:03.950073] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT APPEND COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:57:03.962274] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:57:03.962319] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=26] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:57:03.962342] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=13] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:57:03.962471] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [355646][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=14] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:3, threads_idle_time:1312809374449, tenant_id:500}, cur_thread_count=3, cnt=1, new_thread_count=4) [2025-03-17 09:57:03.962638] WDIAG [LIB] try_wait (thread.cpp:249) [355646][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=30][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=110, ret=-4023) [2025-03-17 09:57:03.962657] INFO [LIB] do_thread_recycle (threads.cpp:146) [355646][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=18] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7aeb5bf7b0) [2025-03-17 09:57:03.962691] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [355646][IO_SYNC_CH2][T0][Y0-0000000000000000-0-0] [lt=33] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:4, threads_idle_time:1312809374449, tenant_id:500}, cur_thread_count=3, cnt=1, new_thread_count=4) [2025-03-17 09:57:03.983476] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:03.984513] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0941-00063020D37365F9-0-0] [lt=2][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.984720] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859731194}], scn_array=[cnt:1, {val:1742176623934963000, v:0}], curr_size=588, accum_size=2350, time_cost=49423) [2025-03-17 09:57:03.984795] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=45][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463552314}], scn_array=[cnt:1, {val:1742176623949971000, v:0}], curr_size=587, accum_size=4384, time_cost=34717) [2025-03-17 09:57:03.985297] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=36][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422178}], scn_array=[cnt:1, {val:1742176623662393002, v:0}], curr_size=122, accum_size=732, time_cost=22914) [2025-03-17 09:57:03.985345] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=51][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708622}], scn_array=[cnt:1, {val:1742176623662393002, v:0}], curr_size=122, accum_size=488, time_cost=22963) [2025-03-17 09:57:03.985305] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B0941-00063020D37365F9-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8b0980, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222517351, assoc_session_id:3222517351, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623979822, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623979821, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222517351, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace8b0980}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:03.986619] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063020D2AB51D5-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:03.986972] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [357553][T1_L0_G19][T1][YB420A0B0940-00063020D2AB51D5-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace8cae60, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222511659, assoc_session_id:3222511659, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176623985160, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176623984095, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7888660038, id:3222511659, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ace8cae60}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:04.020720] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=17] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40422558924, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:04.022700] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=34] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=589824, evict_batch_count=0, elapse_time=0, size_used=131072, mem_used=241250304) [2025-03-17 09:57:04.022755] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=22] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:04.041475] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117896][T1_L0_G0][T1][YB420A0B093F-00062FB7B25CA589-0-0] [lt=22][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:04.042013] INFO [PALF] next (palf_iterator.h:183) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=6] [PALF STAT ITERATOR INFO](io_ctx={user:"ARCHIVE", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:1, miss_cnt:0, cache_read_size:166508, hit_ratio:"1.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:872542236}}}) [2025-03-17 09:57:04.043039] INFO [ARCHIVE] statistic (ob_archive_fetcher.cpp:1068) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=49] archive_fetcher statistic in 10s(total_read_log_size=166508, total_read_cost_ts=1067, total_read_task_count=1, avg_task_lsn_size=166508, avg_task_cost_ts=1067) [2025-03-17 09:57:04.043064] INFO [ARCHIVE] build_send_task_ (ob_archive_fetcher.cpp:869) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=23] build send task succ(id={id:1}, station={key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}) [2025-03-17 09:57:04.043075] INFO [ARCHIVE] back_fill (ob_archive_task.cpp:137) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=11] print back fill task succ(cur_piece={interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset={lsn:872542236}, end_offset={lsn:872708744}, max_scn={val:1742176623662393002, v:0}, this={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}) [2025-03-17 09:57:04.043105] INFO [ARCHIVE] update_log_fetch_task_ (ob_archive_fetcher.cpp:813) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=29] back fill log fetch task succ(fetch_task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}) [2025-03-17 09:57:04.043125] INFO [ARCHIVE] push_fetch_log (ob_ls_task.cpp:749) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=19] print push_fetch_log(task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}, this={has_encount_error:false, is_worm:false, max_no_limit_lsn:{lsn:641930252}, max_archived_info:{offset:{lsn:872542236}, scn:{val:1742176503499574003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}}, max_seq_log_offset:{lsn:939466752}, max_fetch_info:{offset:{lsn:872542236}, scn:{val:1742176503499574003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}}, piece_min_lsn:{lsn:763061037}, archive_file_id:14, archive_file_offset:180252, piece_dir_exist:true, wait_send_task_count:1, tasks:{tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}, {tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}}) [2025-03-17 09:57:04.043193] INFO [ARCHIVE] submit_fetch_log_ (ob_archive_fetcher.cpp:845) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=68] push fetch log succ(id={id:1}, &task=0x7f7a9932d280) [2025-03-17 09:57:04.043199] INFO [ARCHIVE] handle_log_fetch_task_ (ob_archive_fetcher.cpp:404) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=5] handle log fetch task succ(id={id:1}) [2025-03-17 09:57:04.043205] INFO [ARCHIVE] get_sorted_fetch_log (ob_ls_task.cpp:192) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=3] the same file(tuple={offset:{lsn:872542236}, scn:{val:1742176503499574003, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}}, tmp_task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}) [2025-03-17 09:57:04.043223] INFO [ARCHIVE] get_sorted_fetch_log_ (ob_archive_fetcher.cpp:946) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=18] get sorted fetch log succ(task={tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, cur_piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, next_piece:{interval_us:86400000000, genesis_scn:{val:18446744073709551615, v:3}, piece_id:0, base_piece_id:0}, base_scn:{val:1742176383843265001, v:0}, start_offset:{lsn:872361984}, end_offset:{lsn:939466752}, cur_offset:{lsn:872708744}, unfinished_data_size:66758008, max_scn:{val:1742176623662393002, v:0}, send_task:{status:0, tenant_id:1004, id:{id:1}, station:{key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}, start_offset:{lsn:872542236}, end_offset:{lsn:872708744}, max_scn:{val:1742176623662393002, v:0}, file_id:0, file_offset:-1, data:G, data_len:166508, this:0x7f791179c030}, this:0x7f7a9932d280}) [2025-03-17 09:57:04.043257] INFO [ARCHIVE] push_task_status (ob_archive_sender.cpp:189) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=29] push succ(task_status=0x7f78b2966500) [2025-03-17 09:57:04.043267] INFO [ARCHIVE] submit_send_task_ (ob_archive_fetcher.cpp:1003) [3539505][T1004_ArcFetche][T1004][YB420A0B093E-00062FB7FB154125-0-0] [lt=9] submit send task succ(send_task=0x7f791179c030) [2025-03-17 09:57:04.043310] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=31][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.043360] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=26][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.043406] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.043452] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=23][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.043511] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=23][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.043570] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=37][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.053435] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=35][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.053510] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=28][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.053547] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=17][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.053626] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=38][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.053683] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=52][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.054956] INFO [ARCHIVE] push_log_ (ob_archive_sender.cpp:749) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=19] push log succ(id={id:1}) [2025-03-17 09:57:04.054977] INFO [ARCHIVE] archive_log_ (ob_archive_sender.cpp:634) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=20] finish task succ(id={id:1}) [2025-03-17 09:57:04.054983] INFO [ARCHIVE] statistic (ob_archive_sender.cpp:972) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=5] archive_sender statistic in 10s(total_send_log_size=166508, total_send_buf_size=166508, total_send_task_count=1, total_send_cost_ts=11691, avg_task_lsn_size=166508, avg_task_buf_size=166508, avg_task_cost_ts=11691) [2025-03-17 09:57:04.054992] INFO [ARCHIVE] handle (ob_archive_sender.cpp:469) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=9] archive log succ(id={id:1}) [2025-03-17 09:57:04.055003] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=7][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.062365] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:04.063139] INFO [SHARE] blacklist_loop_ (ob_server_blacklist.cpp:313) [117559][Blacklist][T0][Y0-0000000000000000-0-0] [lt=37] blacklist_loop exec finished(cost_time=15, is_enabled=true, send_cnt=0) [2025-03-17 09:57:04.063542] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=36][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.063600] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=23][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.063632] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.063724] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=28][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.063761] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=16][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.064471] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=31] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:04.064507] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=23] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:04.065077] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=10][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.072483] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=23] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=0) [2025-03-17 09:57:04.073636] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=27][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.073699] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=26][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.073735] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=17][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.073831] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=30][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.073860] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=25][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.075214] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=25][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.076923] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [3539421][T1003_LCLSender][T1003][Y0-0000000000000000-0-0] [lt=40] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f79a77e82b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:57:04.081094] INFO [LIB] stat (utility.h:1210) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=30] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:57:04.083555] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=35][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.083726] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=24][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.083804] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=23][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.083850] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=32][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.083895] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=18][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.083935] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=25][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.084115] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422300}], scn_array=[cnt:1, {val:1742176623662393003, v:0}], curr_size=122, accum_size=854, time_cost=21660) [2025-03-17 09:57:04.084203] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=21] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=1, avg_log_gen_to_submit_time=4, avg_log_submit_to_flush_time=18503, avg_log_submit_to_slide_time=18517) [2025-03-17 09:57:04.084262] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1342, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:04.084329] INFO [LIB] stat (utility.h:1210) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19801, this=0x7f79ae333c30, extra_info="palf_id:1") [2025-03-17 09:57:04.084343] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=13][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708744}], scn_array=[cnt:1, {val:1742176623662393003, v:0}], curr_size=122, accum_size=610, time_cost=21914) [2025-03-17 09:57:04.084378] INFO [LIB] stat (utility.h:1227) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30, this=0x7f79c85e4530, extra_info="") [2025-03-17 09:57:04.084461] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=32] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=12, this=0x7f79ae3296a0, extra_info="palf_id:1") [2025-03-17 09:57:04.084477] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=16] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=19919, this=0x7f79ae3298d0, extra_info="palf_id:1") [2025-03-17 09:57:04.084492] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=33, this=0x7f79ae333e60, extra_info="palf_id:1") [2025-03-17 09:57:04.085303] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=21][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.093837] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=25][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.093905] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=25][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.093956] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=26][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.093993] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=16][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.094046] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=22][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.095397] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=22][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.099518] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=34] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=7759, avg_group_log_size=646) [2025-03-17 09:57:04.099555] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=32] [PALF STAT APPEND COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:57:04.103956] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=48][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.104005] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=28][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.104053] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=28][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.104077] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=16][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.104162] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=45][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.105479] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=21][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.114062] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=33][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.114120] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=20][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.114161] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=39][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.114203] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=21][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.114256] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=27][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.115532] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=22][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.119100] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=33] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:04.124161] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=29][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.124218] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=20][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.124270] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=33][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.124307] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=21][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.124353] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=20][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.125033] INFO [LIB] stat (utility.h:1210) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT WRITE LOG COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19397, this=0x7f7adde67c30, extra_info="palf_id:1") [2025-03-17 09:57:04.125070] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=35][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463552901}], scn_array=[cnt:1, {val:1742176623949971001, v:0}], curr_size=122, accum_size=4506, time_cost=43860) [2025-03-17 09:57:04.125096] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=25] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=4506) [2025-03-17 09:57:04.125117] INFO [LIB] stat (utility.h:1227) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=9] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=26, this=0x7f7b067e8d30, extra_info="") [2025-03-17 09:57:04.125135] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=16] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=4506) [2025-03-17 09:57:04.125183] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=34] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=7, this=0x7f7adde5d6a0, extra_info="palf_id:1") [2025-03-17 09:57:04.125194] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=10] [PALF STAT LOG LIFE TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=19512, this=0x7f7adde5d8d0, extra_info="palf_id:1") [2025-03-17 09:57:04.125209] INFO [LIB] stat (utility.h:1210) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=7] [PALF STAT FLUSH CB COST TIME](cur_stat_count=11, stat_interval=1000000, avg (count/cost)=36, this=0x7f7adde67e60, extra_info="palf_id:1") [2025-03-17 09:57:04.125626] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=31][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.130746] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:04.130792] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=22] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:16, fail_bw:65415, [delay/us]:prepare:1, schedule:25, submit:1, rt:1403, total:1431, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:04.130813] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4468, iops:11, bw:49061, [delay/us]:prepare:5, schedule:26, submit:17, rt:19318, total:19366) [2025-03-17 09:57:04.130826] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 10000:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}, group_id = 10000:{mode:2, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:0}]}, hold_mem=31899104, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=2, iops=2, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:04.130849] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=33] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=27036672) [2025-03-17 09:57:04.130879] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=48] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:16, avg_bw:65416, [delay/us]: prepare:1, schedule:25, submit:1, device:1403, total:1431) [2025-03-17 09:57:04.130894] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=6] [IO STATUS FUNCTION](tenant_id=1, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4468, avg_iops:11, avg_bw:49062, [delay/us]: prepare:5, schedule:26, submit:17, device:19318, total:19366) [2025-03-17 09:57:04.130905] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CALLBACK](tenant_id=1, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:340, idx_:0, tid_:117786}, {is_inited_:true, queue_.get_total():0, tg_id_:284, idx_:1, tid_:170341}], io_callback_status=runner 0: thread_id=117786, queue_count=0, runner 1: thread_id=170341, queue_count=0, ) [2025-03-17 09:57:04.130914] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=22] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=65, size_used=0, mem_used=0) [2025-03-17 09:57:04.130938] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=14] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:8, fail_bw:32709, [delay/us]:prepare:1, schedule:26, submit:1, rt:451, total:480, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:04.130955] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS GROUP SYS](tenant_id=1003, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:1, hold_mem:12288 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4437, iops:12, bw:53152, [delay/us]:prepare:5, schedule:19, submit:20, rt:22636, total:22682) [2025-03-17 09:57:04.130953] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=34] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:04.130964] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS TENANT](tenant_id=1003, ref_cnt=4, io_config={group_configs_cnt:3, memory_limit_:1717986918, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=31033536, free_req_cnt=49999, free_result_cnt=49999, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=3, iops=3, ibw=0, obw=0, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:04.130991] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=25] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:8, avg_bw:32710, [delay/us]: prepare:1, schedule:26, submit:1, device:451, total:480) [2025-03-17 09:57:04.131009] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1003, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4437, avg_iops:12, avg_bw:53154, [delay/us]: prepare:5, schedule:19, submit:20, device:22636, total:22682) [2025-03-17 09:57:04.131022] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=1003, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:461, idx_:0, tid_:3539397}], io_callback_status=runner 0: thread_id=3539397, queue_count=0, ) [2025-03-17 09:57:04.131051] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP](tenant_id=1004, io_status=group_id:0, group_name:OTHER_GROUPS, mode:REMOTE WRITE, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:166508, iops:1, bw:166216, limit:9223372036854775807, [delay/us]:prepare:28, schedule:17, submit:21, rt:11484, total:11550) [2025-03-17 09:57:04.131066] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=11] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:SLOG_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:2, bw:8177, [delay/us]:prepare:6, schedule:15, submit:1, rt:19565, total:19587) [2025-03-17 09:57:04.131094] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS GROUP SYS](tenant_id=1004, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4096, iops:20, bw:81776, [delay/us]:prepare:5, schedule:19, submit:140, rt:18734, total:18899) [2025-03-17 09:57:04.131106] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS TENANT](tenant_id=1004, ref_cnt=2, io_config={group_configs_cnt:3, memory_limit_:15461882266, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:8, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:8}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=33369120, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0, 0, 0, 0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=4, iops=4, ibw=0, obw=166216, iops_limit=9223372036854775807, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:04.131142] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=32] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:OTHER_GROUPS, mode:LOCAL WRITE, avg_size:4096, avg_iops:2, avg_bw:8177, [delay/us]: prepare:6, schedule:15, submit:1, device:19565, total:19587) [2025-03-17 09:57:04.131172] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=21] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_MID, mode:REMOTE WRITE, avg_size:166508, avg_iops:1, avg_bw:166212, [delay/us]: prepare:28, schedule:17, submit:21, device:11484, total:11550) [2025-03-17 09:57:04.131181] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS FUNCTION](tenant_id=1004, io_status=function_name:CLOG_HIGH, mode:LOCAL WRITE, avg_size:4096, avg_iops:20, avg_bw:81773, [delay/us]: prepare:5, schedule:19, submit:140, device:18734, total:18899) [2025-03-17 09:57:04.131191] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=5] [IO STATUS CALLBACK](tenant_id=1004, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:399, idx_:0, tid_:3539211}, {is_inited_:true, queue_.get_total():0, tg_id_:407, idx_:1, tid_:3539347}, {is_inited_:true, queue_.get_total():0, tg_id_:408, idx_:2, tid_:3539348}, {is_inited_:true, queue_.get_total():0, tg_id_:409, idx_:3, tid_:3539349}], io_callback_status=runner 0: thread_id=3539211, queue_count=0, runner 1: thread_id=3539347, queue_count=0, runner 2: thread_id=3539348, queue_count=0, runner 3: thread_id=3539349, queue_count=0, ) [2025-03-17 09:57:04.131219] INFO [COMMON] print_io_status (ob_io_manager.cpp:2603) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=17] [IO STATUS GROUP](tenant_id=500, io_status=group_id:0, group_name:OTHER_GROUPS, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]:fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]:size:4096, iops:1, bw:4088, limit:50000, [delay/us]:prepare:0, schedule:0, submit:600000000, rt:3168, total:3176) [2025-03-17 09:57:04.131242] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS TENANT](tenant_id=500, ref_cnt=1, io_config={group_configs_cnt:3, memory_limit_:536870912, callback_thread_count_:0, unit_config_:{min_iops:10000, max_iops:50000, weight:10000, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:100}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}, group_id = 0:{mode:2, deleted:false, cleared:false, min_percent:100, max_percent:100, weight_percent:100}]}, hold_mem=30968256, free_req_cnt=50000, free_result_cnt=50000, callback_queues=[0], [FAILED]: fail_ips=0, fail_ops=0, fail_iops=0, fail_ibw=0, fail_obw=0, [SUCC]: ips=0, ops=0, iops=0, ibw=0, obw=0, iops_limit=50000, ibw_limit=9223372036854775807, obw_limit=9223372036854775807) [2025-03-17 09:57:04.131268] INFO [COMMON] print_io_function_status (ob_io_manager.cpp:2810) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=23] [IO STATUS FUNCTION](tenant_id=500, io_status=function_name:OTHER_GROUPS, mode:LOCAL READ, avg_size:4096, avg_iops:1, avg_bw:4088, [delay/us]: prepare:0, schedule:0, submit:600000000, device:3168, total:3176) [2025-03-17 09:57:04.131286] INFO [COMMON] print_io_status (ob_io_manager.cpp:2752) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CALLBACK](tenant_id=500, runners=[{is_inited_:true, queue_.get_total():0, tg_id_:285, idx_:0, tid_:117626}], io_callback_status=runner 0: thread_id=117626, queue_count=0, ) [2025-03-17 09:57:04.131299] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f7b319af550, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:04.131309] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS CHANNEL] device_id=0x7f798d53e150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131317] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7aa93f0850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131325] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS CHANNEL] device_id=0x7f7a06c3c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131334] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7ad3914150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131345] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=10] [IO STATUS CHANNEL] device_id=0x7f7a39004150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131354] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f79a21c4150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131364] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f79f75bc850, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131375] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7b319ae750, req_in_channel, async_total=0, sync_total=0, async_cnt= 0 0 0 0 0 0 0 0, sync_cnt= 0 [2025-03-17 09:57:04.131384] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f79fed4c150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131393] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7aa93f0150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131402] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f79f75bc150, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131412] INFO [COMMON] print_status (ob_io_struct.cpp:2853) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS CHANNEL] device_id=0x7f7b319b0350, req_in_channel, async_total=0, sync_total=0, async_cnt=, sync_cnt= 0 [2025-03-17 09:57:04.131435] INFO [COMMON] print_server_status (ob_io_manager.cpp:387) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS SERVER] net_in=47977kB/s, net_out=266kB/s, bucket_in=0kB/s, bucket_out=162kB/s, failed_bucket_in=0kB/s, failed_bucket_out=0kB/s, limit=1280000kB/s [2025-03-17 09:57:04.131461] INFO [COMMON] operator() (ob_io_manager.cpp:496) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=8] [IO STATUS BUCKET] storage={1, 1004, 1005}, in=[0 / 0]kB/s, out=[162 / 0]kB/s, ips=[0 / 0], ops=[0 / 0] [2025-03-17 09:57:04.134261] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539500][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAC54125-0-0] [lt=34][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.134323] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539501][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAD54125-0-0] [lt=15][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.134351] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539499][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAB54125-0-0] [lt=10][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.134385] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539503][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAF54125-0-0] [lt=29][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.134483] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539504][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FB054125-0-0] [lt=37][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.135723] WDIAG [ARCHIVE] get_send_task_ (ob_archive_sender.cpp:335) [3539502][T1004_ArcSender][T1004][YB420A0B093E-00062FB7FAE54125-0-0] [lt=17][errcode=0] get task not exist(ret=0, exist=false, task_status={issue:true, ref:2, num:1, id:{id:1}}) [2025-03-17 09:57:04.143637] INFO [ARCHIVE] update_archive_progress (ob_ls_task.cpp:380) [3539257][T1004_ArcSender][T1004][YB420A0B093E-00062FB7F8454125-0-0] [lt=27] update archive progress succ(ret=0, id_={id:1}, station={key:{incarnation:1, dest_id:1005, round:4}, lease:{lease_id:1, lease_start_ts:0, lease_end_ts:0}}, tuple={offset:{lsn:872708744}, scn:{val:1742176623662393002, v:0}, piece:{interval_us:86400000000, genesis_scn:{val:1742011633326698000, v:0}, piece_id:5, base_piece_id:4}}, file_id=14, file_offset=346760) [2025-03-17 09:57:04.146291] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=40][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859731782}], scn_array=[cnt:1, {val:1742176624099192000, v:0}], curr_size=588, accum_size=2938, time_cost=46762) [2025-03-17 09:57:04.148705] INFO [STORAGE.TRANS] print_stat_ (ob_tenant_weak_read_service.cpp:541) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=28] [WRS] [TENANT_WEAK_READ_SERVICE] [STAT](tenant_id=1, server_version={version:{val:1742176623949971000, v:0}, total_part_count:1, valid_inner_part_count:1, valid_user_part_count:0}, server_version_delta=198728, in_cluster_service=true, cluster_version={val:1742176623748781000, v:0}, min_cluster_version={val:1742176623748781000, v:0}, max_cluster_version={val:1742176623748781000, v:0}, get_cluster_version_err=0, cluster_version_delta=399918, cluster_service_master="192.168.1.62:2882", cluster_service_tablet_id={id:226}, post_cluster_heartbeat_count=8524986, succ_cluster_heartbeat_count=14738, cluster_heartbeat_interval=50000, local_cluster_version={val:1742176199632131000, v:0}, local_cluster_delta=424516568, force_self_check=false, weak_read_refresh_interval=100000) [2025-03-17 09:57:04.150966] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:167) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=24] ====== [tabletchange] timer task ======(GC_CHECK_INTERVAL=5000000) [2025-03-17 09:57:04.150997] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=26] [tabletchange] task check ls(ls->get_ls_id()={id:1}, tablet_persist_trigger=0) [2025-03-17 09:57:04.151007] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:201) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=8] [tabletchange] task check ls(ls->get_ls_id()={id:1001}, tablet_persist_trigger=0) [2025-03-17 09:57:04.151012] INFO [STORAGE] runTimerTask (ob_tablet_gc_service.cpp:295) [3539208][T1004_TimerWK2][T1004][Y0-0000000000000000-0-0] [lt=4] [tabletchange] succeed to gc_tablet(ret=0, ret="OB_SUCCESS", ls_cnt=2, times=11654) [2025-03-17 09:57:04.154240] INFO [SERVER] try_reload_schema (ob_server_schema_updater.cpp:449) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=23] schedule fetch new schema task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:04.154259] INFO [SERVER] do_heartbeat_event (ob_heartbeat.cpp:269) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=19] try reload schema success(schema_version=1741661297536408, refresh_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_ret=0) [2025-03-17 09:57:04.154261] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:263) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A3-0-0] [lt=14] [REFRESH_SCHEMA] start to process schema refresh task(ret=0, ret="OB_SUCCESS", schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:04.154283] INFO [SERVER] process_refresh_task (ob_server_schema_updater.cpp:288) [117506][SerScheQueue1][T0][YB420A0B093E-00062FB7F00884A3-0-0] [lt=21] [REFRESH_SCHEMA] local schema info is newer, no need to refresh schema(ret=0, ret="OB_SUCCESS", local_schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}, schema_info={schema_version:1742008591889280, tenant_id:1004, sequence_id:4398046515019}) [2025-03-17 09:57:04.158552] INFO [SHARE] refresh_primary_cluster_id (ob_alive_server_tracer.cpp:400) [117407][TimerWK0][T0][YB420A0B093E-00062FB95017E819-0-0] [lt=43] refresh primary cluster finish(ret=0, primary_cluster_id_=-1) [2025-03-17 09:57:04.162979] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=10] get number(ret=0, service_type_=0, range=1, base_id=1742176624162214000, start_id=1742176624162214000, end_id=1742176624162214001) [2025-03-17 09:57:04.163019] INFO [STORAGE.TRANS] operator() (ob_ts_mgr.h:180) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=23] refresh gts functor(ret=0, ret="OB_SUCCESS", gts_tenant_info={v:1}) [2025-03-17 09:57:04.165046] WDIAG [SERVER] deliver_rpc_request (ob_srv_deliver.cpp:677) [117487][pnio1][T0][YB420A0B093F-00062FB9F07CBE1E-0-0] [lt=35][errcode=-5150] can't deliver request(req={packet:{hdr_:{checksum_:4108948571, pcode_:1861, hlen_:192, priority_:5, flags_:6151, tenant_id_:1005, priv_tenant_id_:0, session_id_:0, trace_id_:12378264242495, timeout_:200000, timestamp:1742176624164894, dst_cluster_id:-1, cost_time:{len:40, arrival_push_diff:0, push_pop_diff:0, pop_process_start_diff:0, process_start_end_diff:0, process_end_response_diff:0, packet_id:97611689, request_arrival_time:0}, compressor_type_:0, original_len_:0, src_cluster_id_:1741272511, seq_no_:1742176624163356, data_version_:0}, chid_:0, clen_:130, assemble:false, msg_count:0, payload:0}, type:0, group:0, sql_req_level:0, connection_phase:0, recv_timestamp_:1742176624165042, enqueue_timestamp_:0, request_arrival_time_:1742176624165042, trace_id_:Y0-0000000000000000-0-0}, ret=-5150) [2025-03-17 09:57:04.165113] WDIAG [SERVER] deliver (ob_srv_deliver.cpp:876) [117487][pnio1][T0][YB420A0B093F-00062FB9F07CBE1E-0-0] [lt=55][errcode=-5150] deliver rpc request fail(&req=0x7f7970610098, ret=-5150) [2025-03-17 09:57:04.174433] INFO [SQL.PC] need_check_schema_version (ob_plan_cache_value.cpp:2334) [118873][T1_L0_G10000][T1004][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=6] need check schema(new_schema_version=1742008591889280, cached_tenant_schema_version=1742008591889280, is_contain_synonym()=false, contain_sys_name_table_=true, is_contain_tmp_tbl()=false, is_contain_sys_pl_object()=false, pc_ctx.sql_ctx_.session_info_->get_has_temp_table_flag()=false, need_check=true, constructed_sql_=SELECT row_id, column_name, column_value FROM __all_core_table WHERE table_name = ? ORDER BY row_id, column_name) [2025-03-17 09:57:04.175126] INFO [DATA_DICT] get_data_dict_meta_info_ (ob_data_dict_meta_info.cpp:482) [3539258][T1004_ArcTimer][T1004][YB420A0B093E-00062FB7F8554125-0-0] [lt=30] get_data_dict_meta_info(tenant_id_=1004, base_scn={val:1742011633326698000, v:0}, record_count=10, valid_record_count=2) [2025-03-17 09:57:04.181064] INFO [SHARE] get_zone_list_ (ob_zone_table_operation.cpp:559) [117778][T1_HBService][T1][YB420A0B093E-00062FB7F27BCA1C-0-0] [lt=23] get inactive zone_list(ret=0, ret="OB_SUCCESS", zone_list=[]) [2025-03-17 09:57:04.182247] INFO do_work (ob_rl_mgr.cpp:710) [117504][rl_mgr0][T0][Y0-0000000000000000-0-0] [lt=25] swc wakeup.(stat_period_=1000000, ready=false) [2025-03-17 09:57:04.183624] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=33][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.184370] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859732370}], scn_array=[cnt:1, {val:1742176624134491000, v:0}], curr_size=588, accum_size=3526, time_cost=37988) [2025-03-17 09:57:04.187139] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=8][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463553023}], scn_array=[cnt:1, {val:1742176624149359000, v:0}], curr_size=587, accum_size=587, time_cost=37371) [2025-03-17 09:57:04.187180] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=9][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872708866}], scn_array=[cnt:1, {val:1742176623662393004, v:0}], curr_size=122, accum_size=732, time_cost=24733) [2025-03-17 09:57:04.187217] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=46][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422422}], scn_array=[cnt:1, {val:1742176623662393004, v:0}], curr_size=122, accum_size=976, time_cost=24745) [2025-03-17 09:57:04.196131] INFO [SHARE] force_set_master_rs (ob_rs_mgr.cpp:277) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC8-0-0] [lt=17] [RS_MGR] force set rs list(master_rs="192.168.1.62:2882") [2025-03-17 09:57:04.196176] INFO [SERVER] handle_heartbeat (ob_service.cpp:3753) [118001][T1_L0_G25][T1][YB420A0B093E-00062FB7F28BCCC8-0-0] [lt=41] handle_heartbeat(ret=0, ret="OB_SUCCESS", hb_request={server:"192.168.1.62:2882", server_id:4, rs_addr:"192.168.1.62:2882", rs_server_status:1, epoch_id:4}, hb_response={zone:"zone1", server:"192.168.1.62:2882", sql_port:2881, build_version:"4.3.5.0_100000202024123117-5d6cb5cbc3f7c1ab6eb22e40abec8e160a8764d5(Dec 31 2024 17:35:01)", start_service_time:1741316885024955, server_health_status:{data_disk_status_:1, data_disk_status:"NORMAL"}}, time_cost=46) [2025-03-17 09:57:04.199727] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=32] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1004, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:57:04.200447] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=28] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:57:04.200519] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=27] inner load succ(latest_freeze_info={frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615679197000, v:0}, frozen_statuses_.count():8, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}]}) [2025-03-17 09:57:04.200544] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:719) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=24] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1742176615679197000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1004, freeze_info:{latest_snapshot_gc_scn:{val:1742176615679197000, v:0}, frozen_statuses_.count():8, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629603390925000, v:0}, schema_version:1741597441416008, data_version:17180067072}, {frozen_version:{val:1741716000068154000, v:0}, schema_version:1741680189951280, data_version:17180067072}, {frozen_version:{val:1741802401720920000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741888803480941000, v:0}, schema_version:1741765243813008, data_version:17180067072}, {frozen_version:{val:1741975200204498000, v:0}, schema_version:1741938766878584, data_version:17180067072}, {frozen_version:{val:1742061601933693000, v:0}, schema_version:1742008591889280, data_version:17180067072}, {frozen_version:{val:1742148003841067000, v:0}, schema_version:1742008591889280, data_version:17180067072}]}}) [2025-03-17 09:57:04.202126] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175476][T1_L0_G5][T1][YB420A0B093F-00062FB725C629BC-0-0] [lt=26] get number(ret=0, service_type_=0, range=1, base_id=1742176624202064000, start_id=1742176624202064000, end_id=1742176624202064001) [2025-03-17 09:57:04.221058] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40435141836, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:04.228928] INFO [COMMON] clean_garbage_node (ob_kvcache_map.cpp:671) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=31] Cache wash clean map node details(ret=0, clean_node_count=1, clean_time=0, clean_start_pos=6200000, clean_num=200000) [2025-03-17 09:57:04.229249] INFO eloop_run (eloop.c:145) [117494][pnio2][T0][Y0-0000000000000000-0-0] [lt=30] PNIO [ratelimit] time: 1742176624229247, bytes: 0, bw: 0.000000 MB/s, add_ts: 1007597, add_bytes: 0 [2025-03-17 09:57:04.246886] WDIAG [SHARE] get (ob_global_stat_proxy.cpp:529) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=37][errcode=-4152] get int failed(name="finish_data_version", ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:04.246912] WDIAG [SHARE] get_finish_data_version (ob_global_stat_proxy.cpp:296) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4152] get failed(ret=-4152, ret="OB_ERR_NULL_VALUE") [2025-03-17 09:57:04.247377] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=7][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.63:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176622159882000, replayable_scn:1742176622159882000, readable_scn:1742176622134336000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176623420461000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:04.247418] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=40][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.64:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176622159882000, replayable_scn:1742176622159882000, readable_scn:1742176622134336000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176623420461000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:04.247462] WDIAG [RPC] wait (ob_async_rpc_proxy.h:453) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=44][errcode=0] execute rpc failed(rc=-5150, server="192.168.1.65:2882", timeout=200000, packet code=1861, arg={tenant_id:1004, tenant_info:{tenant_id:1004, tenant_role:"PRIMARY", switchover_status:"NORMAL", switchover_epoch:0, sync_scn:1742176622159882000, replayable_scn:1742176622159882000, readable_scn:1742176622134336000, recovery_until_scn:4611686018427387903, log_mode:"ARCHIVELOG", max_ls_id:1001, restore_data_mode:{restore data mode:"NORMAL"}}, ora_rowscn:1742176623420461000, finish_data_version:0, data_version_barrier_scn:{val:0, v:0}}, ret=0) [2025-03-17 09:57:04.248538] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539173][T1004_TenantInf][T1003][YB420A0B093E-00062FB9526ACFC8-0-0] [lt=13][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:04.248585] WDIAG [SERVER] get_int (ob_inner_sql_result.cpp:502) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4152] get obj error(ret=-4152) [2025-03-17 09:57:04.248596] WDIAG [SQL] build_service_name_ (ob_service_name_proxy.cpp:586) [3539173][T1004_TenantInf][T1004][Y0-0000000000000000-0-0] [lt=10][errcode=-4152] fail to get column in row. (column_name="service_name_id", ret=-4152) [2025-03-17 09:57:04.253785] WDIAG [OCCAM] runTimerTask (ob_occam_timer.h:163) [3539265][T1004_MultiVers][T1004][Y0-0000000000000000-0-0] [lt=51][errcode=0] task scheduled too delay(scheduled_delay=1189782, expected_run_ts_us_=1742176623064000, ObClockGenerator::getRealClock()=1742176624253783, ret=0, ret="OB_SUCCESS", *this={this:0x7f7ad29facb0, caller:ob_multi_version_garbage_collector.cpp:start:116, function_type:ZN9oceanbase19concurrency_control30ObMultiVersionGarbageCollector5startEvE6$_1073, timer_running_flag:1, total_running_count:1, func_shared_ptr_.ptr:0x7f7ad29faf90, time_wheel:0x7f7b12913990, thread_pool:0x7f7b12913170, is_running:1, time_interval:60.00s, expected_run_time:09:57:03.640, task_priority:1, with_handle_protected:1}) [2025-03-17 09:57:04.253946] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:300) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=10] ====== check clog disk timer task ====== [2025-03-17 09:57:04.253963] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:197) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=12] skip repeat study(ret=0, this={this:0x7f79c75f5030, last_study_timestamp:1742176144227996, last_refresh_timestamp:1742176564250851, last_reclaim_timestamp:1742176144229934, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176143516287002, v:0}, is_inited:true}, current_timestamp=1742176624253960, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:57:04.253982] INFO [PALF] get_disk_usage (palf_env_impl.cpp:920) [117784][T1_TimerWK3][T1][Y0-0000000000000000-0-0] [lt=31] get_disk_usage(ret=0, capacity(MB):=8192, used(MB):=5274) [2025-03-17 09:57:04.254897] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=28] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1742176144219083000, snapshot_type=1, create_time=1742176144220824, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.254925] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=26] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1742176143516287002, snapshot_type=2, create_time=1742176144220824, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.254936] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=9] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=1742176144017344000, snapshot_type=3, create_time=1742176144220824, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.254949] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=11] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1004 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176144220824, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.255064] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:620) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=10] succeed to update global reserved snapshot(ret=0, this={this:0x7f79c75f5030, last_study_timestamp:1742176144227996, last_refresh_timestamp:1742176564250851, last_reclaim_timestamp:1742176144229934, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176143516287002, v:0}, is_inited:true}, global_reserved_snapshot_={val:1742176143516287002, v:0}, reserved_snapshot={val:1742176143516287002, v:0}) [2025-03-17 09:57:04.255082] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:554) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=17] multi-version garbage collector refresh successfully(this={this:0x7f79c75f5030, last_study_timestamp:1742176144227996, last_refresh_timestamp:1742176564250851, last_reclaim_timestamp:1742176144229934, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176143516287002, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1742176143516287002, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1742176144220824, reserved_addr:"192.168.1.62:2882", is_this_server_disabled:false, status:0}) [2025-03-17 09:57:04.255102] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:232) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=19] repeat refresh successfully(ret=0, this={this:0x7f79c75f5030, last_study_timestamp:1742176144227996, last_refresh_timestamp:1742176624255102, last_reclaim_timestamp:1742176144229934, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176143516287002, v:0}, is_inited:true}, current_timestamp=1742176624253990, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2025-03-17 09:57:04.255110] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:267) [3539264][T1004_Occam][T1004][Y0-0000000000000000-0-0] [lt=7] skip repeat reclaim(ret=0, this={this:0x7f79c75f5030, last_study_timestamp:1742176144227996, last_refresh_timestamp:1742176624255102, last_reclaim_timestamp:1742176144229934, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176143516287002, v:0}, is_inited:true}, current_timestamp=1742176624255109, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:57:04.277034] INFO [COMMON] replace_fragment_node (ob_kvcache_map.cpp:728) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=40] Cache replace map node details(ret=0, replace_node_count=0, replace_time=0, replace_start_pos=12200000, replace_num=100000) [2025-03-17 09:57:04.277064] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=29] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=10) [2025-03-17 09:57:04.281926] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422544}], scn_array=[cnt:1, {val:1742176624163293000, v:0}], curr_size=122, accum_size=1098, time_cost=19393) [2025-03-17 09:57:04.283690] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=34][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.299105] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=13] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=7759) [2025-03-17 09:57:04.299572] INFO [COMMON] handle (ob_io_struct.cpp:3072) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78628-0-0] [lt=1] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9420, ref_cnt_:1, raw_buf_:0x7f7a60290150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB93AC78628-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a60280080, time_log:{begin_ts:1742176624298991, enqueue_used:2, dequeue_used:39, submit_used:2, return_used:522, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:04.299633] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB93AC78628-0-0] [lt=1][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.299675] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78628-0-0] [lt=6][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:04.300099] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117750][T1_COMMONLSSe][T1][YB420A0B093E-00062FB93AC78628-0-0] [lt=0] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ade8a26e0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222522828, assoc_session_id:3222522828, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176624298524, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176624297465, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7ade0ccf58, id:3222522828, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ade8a26e0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.309896] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [3539406][T1003_LogLoop][T1003][Y0-0000000000000000-0-0] [lt=28] LogLoopThread round_cost_time(us)(round_cost_time=13) [2025-03-17 09:57:04.315025] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=23] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:929, free_obj_hold_cnt:159, allocator used:4186624, allocator total:4439040}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:149, free_obj_hold_cnt:931, allocator used:70838280, allocator total:70838280}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():0, full allocator used:0, full allocator total:0}, tablets_mem=75277320, tablets_mem_limit=1288490180, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:33, free_obj_hold_cnt:50, allocator used:169984, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=1078) [2025-03-17 09:57:04.316582] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:254) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=14] ====== traversal_flush timer task ====== [2025-03-17 09:57:04.316699] INFO [MDS] flush (mds_table_mgr.cpp:195) [3539395][T1003_TimerWK3][T1003][YB420A0B093E-00062FB7F98E9372-0-0] [lt=23] [FREEZE]no need do flush cause min_rec_scn is larger than freezing scn(ret=0, ret="OB_SUCCESS", ls_->get_ls_id()={id:1}, recycle_scn={val:4611686018427387903, v:0}, need_freeze=false, order_flusher_for_some={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742157237104149001, v:0}}, }, max_consequent_callbacked_scn={val:1742176624298524000, v:0}, *this={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:04.316763] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:289) [3539395][T1003_TimerWK3][T1003][YB420A0B093E-00062FB7F98E9372-0-0] [lt=61] succeed to traversal_flush(ret=0, ls_cnt=1) [2025-03-17 09:57:04.320377] INFO eloop_run (eloop.c:145) [117487][pnio1][T0][Y0-0000000000000000-0-0] [lt=10] PNIO [ratelimit] time: 1742176624320375, bytes: 2535211692928, bw: 46.622633 MB/s, add_ts: 1006005, add_bytes: 49180943 [2025-03-17 09:57:04.321806] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=34][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=309, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.321846] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=40][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13701][2nd=13702]}, offset=1979029) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.321867] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.321878] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9978, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606860677, second_id:13702, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FEC6D3DC-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1979029, size_:309, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7988180080, time_log:{begin_ts:1742176624321261, enqueue_used:3, dequeue_used:22, submit_used:1, return_used:491, callback_enqueue_used:2, callback_dequeue_used:22, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.321926] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=42][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9978, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860677, second_id:13702, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FEC6D3DC-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1979029, size_:309, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7988180080, time_log:{begin_ts:1742176624321261, enqueue_used:3, dequeue_used:22, submit_used:1, return_used:491, callback_enqueue_used:2, callback_dequeue_used:22, callback_finish_used:-1, end_used:1742176624321922}}, part_id:-1}) [2025-03-17 09:57:04.322291] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=31][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=309, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.322326] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=35][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:04.322334] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=8][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7974097295, src_buf_size=309) [2025-03-17 09:57:04.322342] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9810, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606860677, second_id:13702, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FEC6D3DC-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1979029, size_:309, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7988180080, time_log:{begin_ts:1742176624321985, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:272, callback_enqueue_used:0, callback_dequeue_used:8, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.322393] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB7FEC6D3DC-0-0] [lt=44][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9810, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860677, second_id:13702, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FEC6D3DC-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1979029, size_:309, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7988180080, time_log:{begin_ts:1742176624321985, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:272, callback_enqueue_used:0, callback_dequeue_used:8, callback_finish_used:-1, end_used:1742176624322387}}, part_id:-1}) [2025-03-17 09:57:04.327369] INFO [SHARE] load_can_execute_task (ob_balance_task_table_operator.cpp:842) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7E-0-0] [lt=8] load can-execute balance task(ret=0, ret="OB_SUCCESS", task_array=[], sql=select * from __all_balance_task where parent_list is null or parent_list = '') [2025-03-17 09:57:04.327398] INFO [BALANCE] do_work (ob_balance_task_execute_service.cpp:172) [3539179][T1004_BalanceEx][T1004][YB420A0B093E-00062FB7FCFE1A7E-0-0] [lt=28] [BALANCE_EXECUTE] finish one round(ret=0, ret="OB_SUCCESS", task_array_=[], idle_time_us=1000000, task_comment_=) [2025-03-17 09:57:04.337953] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2747) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", [SlidingWindow]={palf_id:1, self:"192.168.1.62:2882", lsn_allocator:{max_log_id:5848327, max_lsn:872709110, max_scn:{val:1742176624163293000, v:0}}, group_buffer:{log_group_buffer: start_lsn:{lsn:0}, reuse_lsn:{lsn:872709110}, reserved_buffer_size:41943040, available_buffer_size:33554432, readable_begin_lsn:{lsn:830766070}}, last_submit_lsn:{lsn:872708988}, last_submit_end_lsn:{lsn:872709110}, last_submit_log_id:5848327, last_submit_log_pid:1, max_flushed_lsn:{lsn:872708988}, max_flushed_end_lsn:{lsn:872709110}, max_flushed_log_pid:1, committed_end_lsn:{lsn:872709110}, last_slide_log_id:5848327, last_slide_scn:{val:1742176624163293000, v:0}, last_slide_lsn:{lsn:872708988}, last_slide_end_lsn:{lsn:872709110}, last_slide_log_pid:1, last_slide_log_accum_checksum:3365278380, last_fetch_end_lsn:{lsn:18446744073709551615}, last_fetch_max_log_id:-1, last_fetch_committed_end_lsn:{lsn:18446744073709551615}, last_truncate_lsn:{lsn:18446744073709551615}, last_fetch_req_time:0, is_truncating:false, is_rebuilding:false, last_rebuild_lsn:{lsn:18446744073709551615}, freeze_mode:"FEEDBACK_FREEZE_MODE", has_pending_handle_submit_task:false, last_fetch_trigger_type:"LOG_LOOP_TH", this:0x7f79ae328530}, [StateMgr]={this:0x7f79ae32c830, self:"192.168.1.62:2882", palf_id:1, role:"LEADER", replica_type:"NORMAL_REPLICA", state:"ACTIVE", prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, leader:"192.168.1.62:2882", leader_epoch:0, is_sync_enabled:true, allow_vote:true, allow_vote_persisted:true, pending_end_lsn:{lsn:18446744073709551615}, scan_disk_log_finished:true, last_check_start_id:5848328, is_changing_config_with_arb:false, reconfirm_start_time_us:1741595957457932, palf_role_change_cb:0x7f79ae3332f0, allow_vote:true}, [ConfigMgr]={palf_id:1, self:"192.168.1.62:2882", alive_paxos_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, alive_paxos_replica_num:1, log_ms_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, running_args:{server:{server:"0.0.0.0:0", timestamp:-1, flag:0}, curr_member_list:0, curr_replica_num:0, new_replica_num:1, config_version:{proposal_id:9223372036854775807, config_seq:-1}, ref_scn:{val:18446744073709551615, v:3}, lock_owner:-1, lock_type:0, added_list:0, removed_list:0, type:"STARTWORKING"}, state:0, checking_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, reconfig_barrier:{prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_end_lsn:{lsn:0}, prev_mode_pid:1}, persistent_config_version:{proposal_id:1, config_seq:2}, ms_ack_list:0, resend_config_version:{proposal_id:1, config_seq:2}, resend_log_list:{learner_num:0, learner_array:[]}, last_submit_config_log_time_us:-1, need_change_config_bkgd:false, bkgd_config_version:{proposal_id:9223372036854775807, config_seq:-1}, region:"sys_region", register_time_us:-1, parent:"0.0.0.0:0", parent_keepalive_time_us:-1, last_submit_register_req_time_us:-1, children:{learner_num:0, learner_array:[]}, last_submit_keepalive_time_us:-1, this:0x7f79ae329c70}, [ModeMgr]={palf_id:1, self:"192.168.1.62:2882", applied_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, accepted_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, last_submit_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}, state:"MODE_INIT", new_proposal_id:9223372036854775807, local_max_lsn:{lsn:18446744073709551615}, local_max_log_pid:9223372036854775807, max_majority_accepted_pid:9223372036854775807, max_majority_lsn:{lsn:18446744073709551615}, max_majority_accepted_mode_meta:{version:-1, proposal_id:9223372036854775807, mode_version:9223372036854775807, access_mode:0, ref_scn:{val:18446744073709551615, v:3}}, follower_list:0, ack_list:0, majority_cnt:9223372036854775807, last_submit_req_ts:-1, resend_mode_meta_list:{learner_num:0, learner_array:[]}}, [LogEngine]={palf_id:1, is_inited:true, min_block_max_scn:{val:18446744073709551615, v:3}, min_block_id:0, min_block_min_scn:{val:2, v:0}, base_lsn_for_block_gc:{lsn:805257216}, log_meta:{version:1, log_prepare_meta:{version:1, voted_for:"0.0.0.0:0", log_proposal_id:1}, log_config_meta:{version:3, proposal_id:1, prev:{version:1, config:{config_version:{proposal_id:0, config_seq:1}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, curr:{version:1, config:{config_version:{proposal_id:1, config_seq:2}, log_sync_memberlist:1{server:"192.168.1.62:2882", timestamp:1, flag:0}, log_sync_replica_num:1, arbitration_member:{server:"0.0.0.0:0", timestamp:-1, flag:0}, learnerlist:{learner_num:0, learner_array:[]}, degraded_learnerlist:{learner_num:0, learner_array:[]}}, lock_meta:{version:1, lock_type:0, lock_owner:-1, lock_time:-1}}, prev_log_proposal_id:9223372036854775807, prev_lsn:{lsn:0}, prev_mode_pid:1}, log_snapshot_meta:{version:2, base_lsn:{lsn:805257216}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, log_replica_property_meta:{allow_vote:true, replica_type:"NORMAL_REPLICA"}, log_mode_meta:{version:1, proposal_id:1, mode_version:0, access_mode:1, ref_scn:{val:1, v:0}}}, log_meta_storage:{log_tail:{lsn:53248}, readable_log_tail:{lsn:53248}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:939466752}, min_scn:{val:0, v:0}, curr_block_id:0, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1/meta", dir_fd:264, min_block_id:0, max_block_id:1, curr_writable_block_id:0}, logical_block_size_:67104768, curr_block_writable_size_:67051520, block_header_serialize_buf_:0x7f79ae32ea38, flashback_version:0}, log_storage:{log_tail:{lsn:872709110}, readable_log_tail:{lsn:872709110}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:872361984}, min_scn:{val:1742176383843265001, v:0}, curr_block_id:13, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1004/1/log", dir_fd:275, min_block_id:0, max_block_id:14, curr_writable_block_id:13}, logical_block_size_:67104768, curr_block_writable_size_:66757642, block_header_serialize_buf_:0x7f79ae330738, flashback_version:0}, palf_epoch:1, last_purge_throttling_ts:1741595957457415, this:0x7f79ae32cf70}, [Reconfirm]={palf_id:1, self:"192.168.1.62:2882", state:"FINISHED", new_proposal_id:1, prepare_log_ack_list:0, curr_paxos_follower_list:0, majority_cnt:1, majority_max_log_server:"192.168.1.62:2882", majority_max_accept_pid:9223372036854775807, majority_max_lsn:{lsn:0}, saved_end_lsn:{lsn:0}, last_submit_prepare_req_time_us:1741595957458223, last_fetch_log_time_us:-1, last_record_sw_start_id:-1, last_notify_fetch_time_us:-1, last_purge_throttling_time_us:-1, this:0x7f79ae32c9b0}) [2025-03-17 09:57:04.338119] INFO [PALF] check_and_switch_state (palf_handle_impl.cpp:2751) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=161] [PALF_DUMP](palf_id=1, self="192.168.1.62:2882", ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176624265893, last_flushed_end_lsn:{lsn:872709110}}]) [2025-03-17 09:57:04.338135] INFO [PALF] report_log_task_trace (log_sliding_window.cpp:4325) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=14] current log_task status(palf_id=1, self="192.168.1.62:2882", log_id=5848328, log_task={header:{begin_lsn:{lsn:18446744073709551615}, end_lsn:{lsn:18446744073709551615}, log_id:-1, min_scn:{val:18446744073709551615, v:3}, max_scn:{val:18446744073709551615, v:3}, data_len:0, proposal_id:9223372036854775807, prev_lsn:{lsn:18446744073709551615}, prev_proposal_id:9223372036854775807, committed_end_lsn:{lsn:18446744073709551615}, data_checksum:-1, accum_checksum:-1, is_padding_log:false, is_raw_write:false}, state_map:{val:0}, ref_cnt:0, gen_ts:-1, freeze_ts:-1, submit_ts:-1, flushed_ts:-1, gen_to_freeze cost time:0, gen_to_submit cost time:0, submit_to_flush cost time:0}, ack_info_list=[{member:{server:"192.168.1.62:2882", timestamp:1, flag:0}, last_ack_time_us:1742176624265893, last_flushed_end_lsn:{lsn:872709110}}]) [2025-03-17 09:57:04.348193] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539254][T1004_LSArchive][T1004][YB420A0B093E-00062FB7F8154125-0-0] [lt=23] gc stale ls task succ [2025-03-17 09:57:04.349590] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:7, job_name_:"SUNDAY_WINDOW", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742076000000980, start_date_:1742076000000000, last_date_:0, this_date_:1742076000000980, next_date_:1742680800000000, end_date_:64060560000000000, total_:0, interval_:"FREQ=WEEKLY; INTERVAL=1", repeat_interval_:"FREQ=WEEKLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.GATHER_DATABASE_STATS_JOB_PROC(72000000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:72000, interval_ts_:604800000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:04.351209] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859733546}], scn_array=[cnt:1, {val:1742176624335725000, v:0}], curr_size=587, accum_size=4701, time_cost=15200) [2025-03-17 09:57:04.351336] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=30][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.351374] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=37][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.351395] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.351406] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7218, ref_cnt_:1, raw_buf_:0x7f7aa4262150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63412-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624350957, enqueue_used:2, dequeue_used:23, submit_used:2, return_used:335, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.351490] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=73][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c7218, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63412-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624350957, enqueue_used:2, dequeue_used:23, submit_used:2, return_used:335, callback_enqueue_used:3, callback_dequeue_used:11, callback_finish_used:-1, end_used:1742176624351481}}, part_id:-1}) [2025-03-17 09:57:04.351757] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=33][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.351776] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=18][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:04.351785] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=8][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f797409715a, src_buf_size=1611) [2025-03-17 09:57:04.351794] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6960, ref_cnt_:1, raw_buf_:0x7f7974096150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63412-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624351512, enqueue_used:1, dequeue_used:13, submit_used:0, return_used:209, callback_enqueue_used:2, callback_dequeue_used:19, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.351829] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=27][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c6960, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63412-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624351512, enqueue_used:1, dequeue_used:13, submit_used:0, return_used:209, callback_enqueue_used:2, callback_dequeue_used:19, callback_finish_used:-1, end_used:1742176624351822}}, part_id:-1}) [2025-03-17 09:57:04.351932] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=1] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222511372, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf37af00}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:04.351973] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=31] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4254}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634425} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:04.352002] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=26] end stmt(ret=0, tx_id=9634425, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf37af00, tx_id:{txid:9634425}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222511372, assoc_session_id:3222511372, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176624349548, active_ts:1742176624349548, commit_ts:-1, finish_ts:-1, timeout_us:29999992, lock_timeout_us:-1, expire_ts:1742176654349540, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468465138332, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624347425}], exec_info_reap_ts:1742176624347424, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624347421, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222511372, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf37af00}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.352112] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634425}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176654349540, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624350609}{self_ls_id:{id:1}, session_id:3222511372, associated_session_id:3222511372, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634425}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468465138332, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624350609, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634425} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.352199] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=78] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634425}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176654349540, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624350609}{self_ls_id:{id:1}, session_id:3222511372, associated_session_id:3222511372, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634425}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468465138332, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624350609, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634425} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=152 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.352265] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=62] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634425}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176624352110, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:04.352287] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=20] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf37af00, tx_id:{txid:9634425}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222511372, assoc_session_id:3222511372, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176624349548, active_ts:1742176624349548, commit_ts:-1, finish_ts:1742176624351674, timeout_us:29999992, lock_timeout_us:-1, expire_ts:1742176654349540, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468465138332, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624347425}], exec_info_reap_ts:1742176624347424, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624347421, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:04.352369] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63412-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:04.363304] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=44][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463553732}], scn_array=[cnt:1, {val:1742176624349548000, v:0}], curr_size=586, accum_size=1295, time_cost=13267) [2025-03-17 09:57:04.363403] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [117790][T1_LogIOCb0][T1][Y0-0000000000000000-0-0] [lt=6] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=5, avg_log_gen_to_submit_time=17, avg_log_submit_to_flush_time=20349, avg_log_submit_to_slide_time=20365) [2025-03-17 09:57:04.363514] INFO [PALF] read (log_cache.cpp:103) [117793][T1_LogSharedQue][T1][Y0-0000000000000000-0-0] [lt=22] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=4081, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:04.383625] INFO [LIB] stat (utility.h:1210) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=53] [PALF STAT WRITE LOG COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17719, this=0x7f79c4aa5c30, extra_info="palf_id:1001") [2025-03-17 09:57:04.383664] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422666}], scn_array=[cnt:1, {val:1742176624163293001, v:0}], curr_size=122, accum_size=1220, time_cost=21014) [2025-03-17 09:57:04.383741] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=75] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1220) [2025-03-17 09:57:04.383764] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.383773] INFO [LIB] stat (utility.h:1227) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=24] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=30, this=0x7f79c85e4c70, extra_info="") [2025-03-17 09:57:04.383791] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=7] [PALF STAT COMMITTED LOG SIZE](palf_id=1001, self="192.168.1.62:2882", committed size=1220) [2025-03-17 09:57:04.383828] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=10, this=0x7f79c4a9b6a0, extra_info="palf_id:1001") [2025-03-17 09:57:04.383840] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=11] [PALF STAT LOG LIFE TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=17832, this=0x7f79c4a9b8d0, extra_info="palf_id:1001") [2025-03-17 09:57:04.383864] INFO [LIB] stat (utility.h:1210) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=8] [PALF STAT FLUSH CB COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=39, this=0x7f79c4aa5e60, extra_info="palf_id:1001") [2025-03-17 09:57:04.391841] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [175477][T1_L0_G5][T1][YB420A0B0941-0006302366DD6B84-0-0] [lt=27] get number(ret=0, service_type_=0, range=1, base_id=1742176624391033000, start_id=1742176624391033000, end_id=1742176624391033001) [2025-03-17 09:57:04.410921] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [3539266][T1004_DBMSSched][T1004][YB420A0B093E-00062FB7FC7B6B06-0-0] [lt=26] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:57:04.415181] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117600][RpcIO][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.415172] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117602][RpcIO][T0][Y0-0000000000000000-0-0] [lt=13] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.415666] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117597][RpcIO][T0][Y0-0000000000000000-0-0] [lt=24] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.415879] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117601][RpcIO][T0][Y0-0000000000000000-0-0] [lt=18] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.416057] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117596][RpcIO][T0][Y0-0000000000000000-0-0] [lt=15] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.416240] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117598][RpcIO][T0][Y0-0000000000000000-0-0] [lt=28] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.416727] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117599][RpcIO][T0][Y0-0000000000000000-0-0] [lt=19] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.429269] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=14] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40424656076, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:04.431946] INFO [SHARE] get_server_tenant_configs (ob_unit_getter.cpp:243) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=11] get_server_tenant_configs(unit_infos=[{unit:unit_id:1, resource_pool_id:1, unit_group_id:1, zone:"zone1", server:"192.168.1.62:2882", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:4, max_cpu:4, memory_size:"5GB", log_disk_size:"8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:4, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:4, }}, pool:{resource_pool_id:1, name:"sys_pool", unit_count:1, unit_config_id:1, zone_list:["zone1"], tenant_id:1, replica_type:0}}, {unit:unit_id:1004, resource_pool_id:1002, unit_group_id:1004, zone:"zone1", server:"192.168.1.62:2882", migrate_from_server:"0.0.0.0:0", is_manual_migrate:false, status:0, replica_type:0, config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:8, max_cpu:8, memory_size:"16GB", log_disk_size:"48GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, pool:{resource_pool_id:1002, name:"pool_general_zone1_ypv", unit_count:1, unit_config_id:1003, zone_list:["zone1"], tenant_id:1004, replica_type:0}}]) [2025-03-17 09:57:04.432008] INFO [SHARE] get_compat_mode (ob_unit_getter.cpp:828) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=58] jx_debug: get tenant compatibility mode(tenant_id=1004, compat_mode=0) [2025-03-17 09:57:04.432017] INFO [SHARE] divide_meta_tenant (ob_unit_resource.cpp:1127) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=7] divide meta tenant resource finish(ret=0, ret="OB_SUCCESS", meta_resource={min_cpu:1, max_cpu:1, memory_size:"1.6GB", log_disk_size:"4.8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }, user_resource={min_cpu:8, max_cpu:8, memory_size:"14.4GB", log_disk_size:"43.2GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }, unit_resource={min_cpu:8, max_cpu:8, memory_size:"16GB", log_disk_size:"48GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }) [2025-03-17 09:57:04.432038] INFO [SHARE] divide_meta_tenant (ob_unit_getter.cpp:134) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=20] divide meta tenant finish(ret=0, ret="OB_SUCCESS", meta_tenant_config={tenant_id:1003, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:1, max_cpu:1, memory_size:"1.6GB", log_disk_size:"4.8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:0, is_removed:false, hidden_sys_data_disk_config_size:0}, user_config={tenant_id:1004, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:8, max_cpu:8, memory_size:"14.4GB", log_disk_size:"43.2GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:0, is_removed:false, hidden_sys_data_disk_config_size:0}) [2025-03-17 09:57:04.433337] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539434][T1003_HAService][T1003][Y0-0000000000000000-0-0] [lt=25] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:04.433914] INFO [SHARE] flush_directive_to_iops_control (ob_resource_plan_manager.cpp:398) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=19] set group iops(ret=0, tenant_id=1, cur_directive.group_id_=10000, cur_io_info={group_name:"ocp_monitor_group", min_percent:0, max_percent:100, weight_percent:0, max_net_bandwidth_percent:100, net_bandwidth_weight_percent:0}) [2025-03-17 09:57:04.433975] INFO [SERVER.OMT] set_cpu_shares_ (ob_cgroup_ctrl.cpp:599) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=25] set cpu shares success, group_path=cgroup//tenant_0001//OBRM_ocp_monitor_group, cpu=0.40, cpu_shares_value=409, tenant_id=1 [2025-03-17 09:57:04.434026] INFO [SERVER.OMT] set_cpu_cfs_quota_ (ob_cgroup_ctrl.cpp:724) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=22] set cpu cfs quota success, tenant_id=1, group_id=10000, cpu=-1.00, target_cpu=-1.00, current_cpu=-1.00, group_path=cgroup//tenant_0001//OBRM_ocp_monitor_group [2025-03-17 09:57:04.434066] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:176) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] refresh resource plan success(tenant_id=1, plan_name=ocp_monitor_plan, directives=[{tenant_id:1, group_name:"ocp_monitor_group", mgmt_p1:"4.000000000000000000e+01", utilization_limit:"8.000000000000000000e+01", min_iops:0, max_iops:100, weight_iops:0, group_id:10000, max_net_bandwidth:100, net_bandwidth_weight:0, level:1}]) [2025-03-17 09:57:04.435574] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:189) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=25] refresh resource user mapping rule(tenant_id=1, plan=ocp_monitor_plan, user_rules=[{tenant_id:1, user_id:500008, group_id:10000, group_name:"ocp_monitor_group"}]) [2025-03-17 09:57:04.436504] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:227) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=25] refresh resource mapping rule(tenant_id=1, plan=ocp_monitor_plan, rules=[]) [2025-03-17 09:57:04.436531] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:149) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=26] refresh resource mapping rule success(tenant_id=1, plan=ocp_monitor_plan) [2025-03-17 09:57:04.437418] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [117690][OmtNodeBalancer][T1][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=7] trace tenant dfc parameters(max_parallel_cnt_=40, max_blocked_buffer_size_=13107200, max_buffer_size_=26214400) [2025-03-17 09:57:04.437482] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [117690][OmtNodeBalancer][T1][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=26] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2025-03-17 09:57:04.437493] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [117690][OmtNodeBalancer][T1][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] tenant dfc status(ret=0, get_tenant_id()=1, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=40, get_max_blocked_buffer_size()=13107200, get_max_buffer_size()=26214400, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2025-03-17 09:57:04.439105] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:176) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=7] refresh resource plan success(tenant_id=1003, plan_name=, directives=[]) [2025-03-17 09:57:04.440116] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:189) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=25] refresh resource user mapping rule(tenant_id=1003, plan=, user_rules=[]) [2025-03-17 09:57:04.441013] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:227) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=15] refresh resource mapping rule(tenant_id=1003, plan=, rules=[]) [2025-03-17 09:57:04.441029] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:149) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=16] refresh resource mapping rule success(tenant_id=1003, plan=) [2025-03-17 09:57:04.441796] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [117690][OmtNodeBalancer][T1003][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=6] trace tenant dfc parameters(max_parallel_cnt_=10, max_blocked_buffer_size_=3276800, max_buffer_size_=6553600) [2025-03-17 09:57:04.441864] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [117690][OmtNodeBalancer][T1003][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=24] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2025-03-17 09:57:04.441875] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [117690][OmtNodeBalancer][T1003][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] tenant dfc status(ret=0, get_tenant_id()=1003, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=10, get_max_blocked_buffer_size()=3276800, get_max_buffer_size()=6553600, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2025-03-17 09:57:04.443452] INFO [SHARE] refresh_resource_plan (ob_resource_plan_manager.cpp:176) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] refresh resource plan success(tenant_id=1004, plan_name=, directives=[]) [2025-03-17 09:57:04.444494] INFO [SHARE] refresh_resource_user_mapping_rule (ob_resource_mapping_rule_manager.cpp:189) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=25] refresh resource user mapping rule(tenant_id=1004, plan=, user_rules=[]) [2025-03-17 09:57:04.444552] INFO [SHARE] get_freeze_info_larger_or_equal_than (ob_freeze_info_proxy.cpp:168) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=23] finish load_freeze_info(ret=0, ret="OB_SUCCESS", tenant_id=1003, sql=SELECT * FROM __all_freeze_info WHERE frozen_scn >= 1 ORDER BY frozen_scn) [2025-03-17 09:57:04.445224] INFO [SHARE] get_all_snapshots (ob_snapshot_table_proxy.cpp:428) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=17] get all snapshots(ret=0, snapshots.count()=0, snapshots=[]) [2025-03-17 09:57:04.445287] INFO [SHARE] update_freeze_info (ob_freeze_info_manager.cpp:219) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=24] inner load succ(latest_freeze_info={frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}, freeze_info_={latest_snapshot_gc_scn:{val:1742176615071995000, v:0}, frozen_statuses_.count():3, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629601710007000, v:0}, schema_version:1741597433156656, data_version:17180067072}, {frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}]}) [2025-03-17 09:57:04.445317] INFO [STORAGE] inner_update_info (ob_tenant_freeze_info_mgr.cpp:719) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=28] ObTenantFreezeInfoMgr success to update infos(new_snapshot_gc_scn={val:1742176615071995000, v:0}, new_freeze_infos=[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629601710007000, v:0}, schema_version:1741597433156656, data_version:17180067072}, {frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}], new_snapshots=[], freeze_info_mgr_={is_inited:true, tenant_id:1003, freeze_info:{latest_snapshot_gc_scn:{val:1742176615071995000, v:0}, frozen_statuses_.count():3, frozen_statuses:[{frozen_version:{val:1, v:0}, schema_version:1, data_version:17180067072}, {frozen_version:{val:1741629601710007000, v:0}, schema_version:1741597433156656, data_version:17180067072}, {frozen_version:{val:1741716004303628000, v:0}, schema_version:1741597433156656, data_version:17180067072}]}}) [2025-03-17 09:57:04.445454] INFO [SHARE] refresh_resource_function_mapping_rule (ob_resource_mapping_rule_manager.cpp:227) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=27] refresh resource mapping rule(tenant_id=1004, plan=, rules=[]) [2025-03-17 09:57:04.445471] INFO [SHARE] refresh_resource_mapping_rule (ob_resource_mapping_rule_manager.cpp:149) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=16] refresh resource mapping rule success(tenant_id=1004, plan=) [2025-03-17 09:57:04.446239] INFO [SQL.DTL] calc_max_buffer (ob_dtl_fc_server.cpp:155) [117690][OmtNodeBalancer][T1004][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=8] trace tenant dfc parameters(max_parallel_cnt_=80, max_blocked_buffer_size_=26214400, max_buffer_size_=52428800) [2025-03-17 09:57:04.446295] INFO [SQL.DTL] auto_free_on_time (ob_dtl_tenant_mem_manager.cpp:232) [117690][OmtNodeBalancer][T1004][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=16] auto free to reserve buffer count(reserve_buffer_min_size=16777216, max_reserve_count=2, buffer_size=65536, ret=0) [2025-03-17 09:57:04.446308] INFO [SQL.DTL] clean_on_timeout (ob_dtl_fc_server.cpp:127) [117690][OmtNodeBalancer][T1004][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=11] tenant dfc status(ret=0, get_tenant_id()=1004, get_channel_cnt()=0, get_current_buffer_used()=0, get_current_blocked_cnt()=0, get_current_buffer_cnt()=0, get_max_parallel()=80, get_max_blocked_buffer_size()=26214400, get_max_buffer_size()=52428800, get_accumulated_blocked_cnt()=0, get_max_size_per_channel()=655360) [2025-03-17 09:57:04.446333] INFO [SERVER.OMT] periodically_check_tenant (ob_tenant_node_balancer.cpp:529) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=16] refresh resource manager plan(plan_mgr=background_quota:2147483647, tenant_plan_map:{1->{value:"ocp_monitor_plan"}, 1003->{value:""}, 1004->{value:""}}, rule_mgr=user_rule_map:{{tenant_id:1, user_id_:500008}->10000} function_rule_map:{} group_id_name_map:{{tenant_id:1, group_id:10000}->{value:"ocp_monitor_group"}} group_name_id_map:{{tenant_id:1, group_name:{value:"ocp_monitor_group"}}->10000}, col_rule_mgr={{v:1}->tenant_id:1, version: 1741314275510586, rule_id_map:{} group_id_map:{} {v:1003}->tenant_id:1003, version: 0, rule_id_map:{} group_id_map:{} {v:1004}->tenant_id:1004, version: 0, rule_id_map:{} group_id_map:{} }) [2025-03-17 09:57:04.446893] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:112) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=559] refresh tenant units(sys_unit_cnt=1, units=[{tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:4, max_cpu:4, memory_size:"5GB", log_disk_size:"8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:4, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:4, }}, mode:0, create_timestamp:1741318291145813, is_removed:false, hidden_sys_data_disk_config_size:0}, {tenant_id:1004, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:8, max_cpu:8, memory_size:"14.4GB", log_disk_size:"43.2GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}, {tenant_id:1003, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:1, max_cpu:1, memory_size:"1.6GB", log_disk_size:"4.8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}], ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:04.446928] INFO set_nway (ob_tenant_mutil_allocator.cpp:392) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=30] finish set nway(tenant_id_=1, nway=4) [2025-03-17 09:57:04.446934] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1, nway=4, new_tma_limit=3221225460, pre_tma_limit=3221225460, cur_memstore_limit_percent=40, tenant_config={tenant_id:1, unit_id:1, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1, name:"sys_unit_config", resource:{min_cpu:4, max_cpu:4, memory_size:"5GB", log_disk_size:"8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:4, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:4, }}, mode:0, create_timestamp:1741318291145813, is_removed:false, hidden_sys_data_disk_config_size:0}) [2025-03-17 09:57:04.446947] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:287) [117690][OmtNodeBalancer][T1][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=11] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=4.915199966430664062e+02, this=0x7f7b0f3da170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=2457, Resource Limit(MB)=2457, Throttle Trigger(MB)=1474, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=4.914307641360822765e-01) [2025-03-17 09:57:04.446960] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [117690][OmtNodeBalancer][T1][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=12] succ to set_memstore_threshold of memstore allocator(tenant_id=1, ret=0) [2025-03-17 09:57:04.446966] INFO set_nway (ob_tenant_mutil_allocator.cpp:392) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=4] finish set nway(tenant_id_=1004, nway=8) [2025-03-17 09:57:04.446971] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=5] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1004, nway=8, new_tma_limit=7730941100, pre_tma_limit=7730941100, cur_memstore_limit_percent=50, tenant_config={tenant_id:1004, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:8, max_cpu:8, memory_size:"14.4GB", log_disk_size:"43.2GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}) [2025-03-17 09:57:04.446987] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:287) [117690][OmtNodeBalancer][T1004][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=15] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.474559993743896484e+03, this=0x7f79c2474170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=7372, Resource Limit(MB)=7372, Throttle Trigger(MB)=4423, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=6.083503486695933209e-03) [2025-03-17 09:57:04.446995] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [117690][OmtNodeBalancer][T1004][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=7] succ to set_memstore_threshold of memstore allocator(tenant_id=1004, ret=0) [2025-03-17 09:57:04.447001] INFO set_nway (ob_tenant_mutil_allocator.cpp:392) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=4] finish set nway(tenant_id_=1003, nway=1) [2025-03-17 09:57:04.447004] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:403) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=3] ObTenantMutilAllocator update tenant mem_limit finished(ret=0, tenant_id=1003, nway=1, new_tma_limit=1030792140, pre_tma_limit=1030792140, cur_memstore_limit_percent=40, tenant_config={tenant_id:1003, unit_id:1004, has_memstore:true, unit_status:"NORMAL", config:{unit_config_id:1003, name:"config_general_zone1_C8G16_ypv", resource:{min_cpu:1, max_cpu:1, memory_size:"1.6GB", log_disk_size:"4.8GB", data_disk_size:0, min_iops:9223372036854775807, max_iops:9223372036854775807, iops_weight:8, max_net_bandwidth:INT64_MAX, net_bandwidth_weight:8, }}, mode:0, create_timestamp:1741595942918766, is_removed:false, hidden_sys_data_disk_config_size:0}) [2025-03-17 09:57:04.447014] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:287) [117690][OmtNodeBalancer][T1003][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=9] [Throttle] Update Throttle Unit Config(is_adaptive_update=false, N=1.310719985961914062e+02, this=0x7f79a7674170, enable_adaptive_limit_=false, Unit Name=Memstore, Config Specify Resource Limit(MB)=655, Resource Limit(MB)=655, Throttle Trigger(MB)=393, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=9.610614047311770491e+01) [2025-03-17 09:57:04.447026] INFO update_tenant_mem_limit (ob_tenant_mutil_allocator_mgr.cpp:412) [117690][OmtNodeBalancer][T1003][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=12] succ to set_memstore_threshold of memstore allocator(tenant_id=1003, ret=0) [2025-03-17 09:57:04.447793] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=3] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447815] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=20] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1001, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447824] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=8] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1002, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447836] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=11] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1003, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447847] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1004, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447858] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1005, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447865] INFO [SERVER.OMT] refresh_tenants (ob_tenant_config_mgr.cpp:225) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=6] [DATA_VERSION] periodically update data_version(tmp_ret=0, tmp_ret="OB_SUCCESS", tenant_id=1006, value_updated=true, data_version={version_str:"4.3.5.0", version_val:17180067072}) [2025-03-17 09:57:04.447860] INFO [SERVER] scheduler (ob_dbms_sched_job_master.cpp:219) [3539266][T1004_DBMSSched][T1004][YB420A0B093E-00062FB7FC7B6B07-0-0] [lt=34] success to scheduler single dbms sched job(ret=0, tmp_ret=0) [2025-03-17 09:57:04.447901] INFO [SERVER] cal_all_part_disk_default_percentage (ob_server_utils.cpp:309) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=7] cal_all_part_disk_default_percentage succ(data_dir="/home/admin/oceanbase/store/sgkydb/sstable", clog_dir="/home/admin/oceanbase/store/sgkydb/clog", shared_mode=true, data_disk_total_size=984727396352, data_disk_default_percentage=60, clog_disk_total_size=984727396352, clog_disk_default_percentage=30) [2025-03-17 09:57:04.447911] INFO [SERVER] decide_disk_size (ob_server_utils.cpp:345) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=10] decide disk size finished(suggested_disk_size=0, suggested_disk_percentage=15, default_disk_percentage=30, total_space=984727396352, disk_size=147709109452) [2025-03-17 09:57:04.447917] INFO [SERVER] get_log_disk_info_in_config (ob_server_utils.cpp:88) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=6] get_log_disk_info_in_config(suggested_data_disk_size=0, suggested_clog_disk_size=0, suggested_data_disk_percentage=0, suggested_clog_disk_percentage=15, log_disk_size=147709109452, log_disk_percentage=0, total_log_disk_size=984727396352) [2025-03-17 09:57:04.447924] INFO [CLOG] try_resize (ob_server_log_block_mgr.cpp:800) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=5] try_resize success(ret=0, log_disk_size=147709109452, total_log_disk_size=984727396352, this={dir::"/home/admin/oceanbase/store/sgkydb/clog/log_pool", dir_fd:16, meta_fd:17, log_pool_meta:{curr_total_size:147706609664, next_total_size:147706609664, status:0}, min_block_id:2437, max_block_id:4414, min_log_disk_size_for_all_tenants_:60129542144, is_inited:true}) [2025-03-17 09:57:04.447937] INFO [SERVER.OMT] run1 (ob_tenant_node_balancer.cpp:136) [117690][OmtNodeBalancer][T0][YB420A0B093E-00062FB7EDF6905F-0-0] [lt=12] refresh tenant config(tenants=[1, 1001, 1002, 1003, 1004, 1005, 1006], ret=0) [2025-03-17 09:57:04.452718] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=10] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:8, job_name_:"OPT_STATS_HISTORY_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742114352860434, start_date_:1741682352858898, last_date_:1742027952860036, this_date_:1742114352860434, next_date_:1742200752858898, end_date_:64060560000000000, total_:1414502, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:0, flag_:0, what_:"DBMS_STATS.PURGE_STATS(NULL)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:43200, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:04.454542] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=30][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.454591] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=47][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.454610] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=18][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.454618] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=6][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9c48, ref_cnt_:1, raw_buf_:0x7f7a2293c150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63413-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176624453968, enqueue_used:2, dequeue_used:41, submit_used:2, return_used:491, callback_enqueue_used:3, callback_dequeue_used:32, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.454663] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=39][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c9c48, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63413-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:true, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:0, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f79f0298080, time_log:{begin_ts:1742176624453968, enqueue_used:2, dequeue_used:41, submit_used:2, return_used:491, callback_enqueue_used:3, callback_dequeue_used:32, callback_finish_used:-1, end_used:1742176624454659}}, part_id:-1}) [2025-03-17 09:57:04.455085] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=30][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:04.455255] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222515525, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3cde60}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:04.455322] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=51] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:2132}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634432} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:04.455360] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=34] end stmt(ret=0, tx_id=9634432, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf3cde60, tx_id:{txid:9634432}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222515525, assoc_session_id:3222515525, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176624453001, active_ts:1742176624453001, commit_ts:-1, finish_ts:-1, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176654452996, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468568066021, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624451941}], exec_info_reap_ts:1742176624451940, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624451937, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222515525, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3cde60}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.455524] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634432}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176654452996, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624453001}{self_ls_id:{id:1}, session_id:3222515525, associated_session_id:3222515525, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634432}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468568066021, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624453001, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634432} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.455637] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=97] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634432}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176654452996, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624453001}{self_ls_id:{id:1}, session_id:3222515525, associated_session_id:3222515525, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634432}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468568066021, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624453001, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634432} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=164 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.455716] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=72] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634432}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176624455510, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:04.455773] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=52] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3cde60, tx_id:{txid:9634432}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222515525, assoc_session_id:3222515525, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176624453001, active_ts:1742176624453001, commit_ts:-1, finish_ts:1742176624455133, timeout_us:29999995, lock_timeout_us:-1, expire_ts:1742176654452996, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468568066021, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624451941}], exec_info_reap_ts:1742176624451940, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624451937, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:04.455882] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63413-0-0] [lt=1][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:04.466386] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [3539260][T1004_HAService][T1004][Y0-0000000000000000-0-0] [lt=22] start do ls ha handler(ls_id_array_=[{id:1001}, {id:1}]) [2025-03-17 09:57:04.477181] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=10] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=9) [2025-03-17 09:57:04.482153] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=24] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:04.482168] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=24] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:04.482193] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=35] temporary file module use no blocks [2025-03-17 09:57:04.482196] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=25] temporary file module use no blocks [2025-03-17 09:57:04.482204] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:04.482202] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=8] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:04.482194] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:179) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=17] tmp file swap statistics(swap_task_cnt=0, avg_swap_response_time=0, max_swap_response_time=-1, min_swap_response_time=-1) [2025-03-17 09:57:04.482216] INFO [STORAGE] print_block_usage (ob_tmp_file_block_manager.cpp:821) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=19] temporary file module use no blocks [2025-03-17 09:57:04.482214] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=10] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=8096, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:04.482213] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=8] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=18975, fat_actual_size=1012, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:04.482224] INFO [STORAGE] print_statistics (ob_tmp_file_thread_job.cpp:125) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=7] tmp file flush statistics(flush_task_cnt=0, avg_flush_data_len=0, max_flush_data_len=-1, min_flush_data_len=-1, f1_cnt=0, f2_cnt=0, f3_cnt=0, f4_cnt=0, f5_cnt=0) [2025-03-17 09:57:04.482227] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539233][T1004_TFSwap][T1004][Y0-0000000000000000-0-0] [lt=13] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863119373937, flush_io_finished_ret_:0, flush_io_finished_round_:580432, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:04.482226] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [117805][T1_TFSwap][T1][Y0-0000000000000000-0-0] [lt=11] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863119373928, flush_io_finished_ret_:0, flush_io_finished_round_:859382, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:04.482237] INFO [STORAGE] print_statistics (ob_tmp_file_write_buffer_pool.cpp:1519) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=13] tmp file write buffer pool statistics(dirty_page_percentage=0, max_page_num=2277, fat_actual_size=253, dirty_page_num=0, total_write_back_num=0, meta_page_num=0, dirty_meta_page_num=0, write_back_meta_num=0, data_page_num=0, dirty_data_page_num=0, write_back_data_num=0, data_page_watermark=0, meta_page_watermark=0) [2025-03-17 09:57:04.482254] INFO [STORAGE] do_work_ (ob_tmp_file_thread_wrapper.cpp:308) [3539415][T1003_TFSwap][T1003][Y0-0000000000000000-0-0] [lt=16] ObTmpFileFlushTG information(this={is_inited_:true, mode_:1, last_flush_timestamp_:863119373963, flush_io_finished_ret_:0, flush_io_finished_round_:580433, flushing_block_num_:0, is_fast_flush_meta_:false, fast_flush_meta_task_cnt_:0, wait_list_size_:0, retry_list_size_:0, finished_list_size_:0, normal_loop_cnt_:2, normal_idle_loop_cnt_:2, fast_loop_cnt_:0, fast_idle_loop_cnt_:0, flush_mgr_:{is_inited_:true, flush_ctx_:{is_inited_:true, fail_too_many_:false, expect_flush_size_:0, actual_flush_size_:0, flush_seq_ctx_:{create_flush_task_cnt_:0, prepare_finished_cnt_:0, flush_sequence_:0}, state_:5, iter_:{is_inited_:false, cur_caching_list_idx_:0, cur_caching_list_is_meta_:false, cur_iter_dir_idx_:-1, cur_iter_file_idx_:-1, cached_file_num_:0, cached_dir_num_:0}, file_ctx_hash_.size():0, flush_failed_array_.size():0}}}) [2025-03-17 09:57:04.483445] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=32] Slow write(duration=18410, write_len=4096) [2025-03-17 09:57:04.483465] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=20] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107791, offset=57289515}, end_cursor:ObLogCursor{file_id=5, log_id=1107792, offset=57289821}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17730050, len:1237, log_data_len:306, seq:1107791, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:04.483615] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=18] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}) [2025-03-17 09:57:04.483622] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=18][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162422788}], scn_array=[cnt:1, {val:1742176624163293002, v:0}], curr_size=122, accum_size=122, time_cost=20843) [2025-03-17 09:57:04.483649] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=21] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176624163293001, v:0}, arg={tenant_id:1004, ls_id:{id:1}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1}, cur_readable_scn:{val:1742176624163293001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:04.483829] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.485465] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=20][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:04.485826] INFO [LIB] stat (utility.h:1210) [3539176][T1004_TeRec][T1003][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=25] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=1, this=0x7f79aaff1470, extra_info="") [2025-03-17 09:57:04.492990] WDIAG [SHARE] refresh (ob_task_define.cpp:402) [117415][LogLimiterRefre][T0][Y0-0000000000000000-0-0] [lt=48][errcode=0] Throttled WDIAG logs in last second(details {error code, dropped logs, earliest tid}=[{errcode:-4016, dropped:580, tid:3539397}]) [2025-03-17 09:57:04.499880] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=31][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859734133}], scn_array=[cnt:1, {val:1742176624484974000, v:0}], curr_size=1132, accum_size=5833, time_cost=13986) [2025-03-17 09:57:04.521567] INFO [STORAGE.REDO] write_logs (ob_storage_log_writer.cpp:512) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=16] Slow write(duration=21399, write_len=4096) [2025-03-17 09:57:04.521604] INFO [STORAGE.REDO] notify_flush (ob_storage_log_writer.cpp:552) [3539232][T1004_OB_SLOG][T1004][Y0-0000000000000000-0-0] [lt=36] Successfully flush(log_item={start_cursor:ObLogCursor{file_id=5, log_id=1107792, offset=57289821}, end_cursor:ObLogCursor{file_id=5, log_id=1107793, offset=57290118}, is_inited:true, is_local:false, buf_size:8192, buf:0x7f7b17738050, len:931, log_data_len:297, seq:1107792, flush_finish:false, flush_ret:0}) [2025-03-17 09:57:04.521636] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=38][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859735265}], scn_array=[cnt:1, {val:1742176624498673000, v:0}], curr_size=588, accum_size=6421, time_cost=21657) [2025-03-17 09:57:04.521763] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3650) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=19] start get_ls_replayed_scn(arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}) [2025-03-17 09:57:04.521802] INFO [SERVER] get_ls_replayed_scn (ob_service.cpp:3733) [355733][T1004_L0_G18][T1004][YB420A0B093E-00062FB7F7ED731F-0-0] [lt=23] finish get_ls_replayed_scn(ret=0, ret="OB_SUCCESS", cur_readable_scn={val:1742176624163293001, v:0}, arg={tenant_id:1004, ls_id:{id:1001}, all_replica:false}, result={tenant_id:1004, ls_id:{id:1001}, cur_readable_scn:{val:1742176624163293001, v:0}, offline_scn:{val:18446744073709551615, v:3}, self_addr_:"192.168.1.62:2882"}, offline_scn={val:18446744073709551615, v:3}) [2025-03-17 09:57:04.522845] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=39] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=589824, evict_batch_count=0, elapse_time=0, size_used=131072, mem_used=241250304) [2025-03-17 09:57:04.522911] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=27] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:04.538081] INFO [PALF] log_loop_ (log_loop_thread.cpp:155) [3539223][T1004_LogLoop][T1004][Y0-0000000000000000-0-0] [lt=37] LogLoopThread round_cost_time(us)(round_cost_time=13) [2025-03-17 09:57:04.546523] INFO [STORAGE.TRANS] dump_mapper_info (ob_lock_wait_mgr.h:67) [3539416][T1003_LockWaitM][T1003][Y0-0000000000000000-0-0] [lt=31] report RowHolderMapper summary info(count=0, bkt_cnt=248) [2025-03-17 09:57:04.549753] WDIAG [SERVER.OMT] get_tenant_base_with_lock (ob_multi_tenant.cpp:261) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] get tenant from omt failed(ret=-5150, tenant_id=1002) [2025-03-17 09:57:04.549787] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=27][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.549826] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=19][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.549848] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.549873] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=11][errcode=-5150] switch tenant fail(tenant_id=1001, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.549914] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.549966] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=38] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=12, avg_log_batch_cnt=1, total_group_log_size=4677, avg_group_log_size=389) [2025-03-17 09:57:04.551255] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=12][errcode=-5150] switch tenant fail(tenant_id=1002, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.551287] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=17][errcode=-5150] switch tenant fail(tenant_id=1005, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.551315] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=15][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.551335] WDIAG [SHARE] switch_to (ob_tenant_base.cpp:600) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=10][errcode=-5150] switch tenant fail(tenant_id=1006, ret=-5150, lbt()="0x810107f 0x1a796460 0x7ac62f1 0x18775210 0x187762d8 0x187755f6 0x1f9588e5 0x1f9566ee 0x7f7b46c081ca 0x7f7b468398d3") [2025-03-17 09:57:04.551841] INFO [STORAGE.TRANS] run1 (ob_black_list.cpp:194) [117927][BlackListServic][T0][Y0-0000000000000000-0-0] [lt=9] ls blacklist refresh finish(cost_time=14697) [2025-03-17 09:57:04.554851] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=0][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.554894] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=41][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.554917] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=22][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.554928] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=10][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8a48, ref_cnt_:1, raw_buf_:0x7f7ac6a9a150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0078-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624554354, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:442, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:04.554982] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=47][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c8a48, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0078-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624554354, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:442, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:-1, end_used:1742176624554976}}, part_id:-1}) [2025-03-17 09:57:04.554988] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=30][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624554354, enqueue_used:2, dequeue_used:21, submit_used:1, return_used:442, callback_enqueue_used:2, callback_dequeue_used:26, callback_finish_used:-1, end_used:1742176624554976}}) [2025-03-17 09:57:04.555046] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=53][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:04.555058] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:04.555413] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=50][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.555452] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=38][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:04.555460] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7aaacc1510, src_buf_size=295) [2025-03-17 09:57:04.555468] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9ff0, ref_cnt_:1, raw_buf_:0x7f7aaacc0170, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0078-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624555070, enqueue_used:1, dequeue_used:30, submit_used:1, return_used:294, callback_enqueue_used:2, callback_dequeue_used:13, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:04.555507] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=33][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9ff0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB7FD3E0078-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624555070, enqueue_used:1, dequeue_used:30, submit_used:1, return_used:294, callback_enqueue_used:2, callback_dequeue_used:13, callback_finish_used:-1, end_used:1742176624555503}}, part_id:-1}) [2025-03-17 09:57:04.555513] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=9][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7ac4e04080, time_log:{begin_ts:1742176624555070, enqueue_used:1, dequeue_used:30, submit_used:1, return_used:294, callback_enqueue_used:2, callback_dequeue_used:13, callback_finish_used:122, end_used:-30}}) [2025-03-17 09:57:04.555547] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=32][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:04.555560] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=11][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac4e04080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.555589] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=29][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7ac4e04080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.555615] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=23][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:04.555633] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=17][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc5d610}) [2025-03-17 09:57:04.555653] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=20][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc5e3b8, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:0, rowkey_end_idx:0, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc5e3b8, micro_handle:NULL}) [2025-03-17 09:57:04.555687] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=33][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:57:04.555695] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=8][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac8638a58, local_datums:0x7f7ac8638a58}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:57:04.555865] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=169][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f7ac8638a58, local_datums:0x7f7ac8638a58}, datums:[]}) [2025-03-17 09:57:04.555881] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=16][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176624548933000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:57:04.555892] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=10][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f7ac8637ac0) [2025-03-17 09:57:04.555900] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:57:04.555908] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1003},{"BIGINT":1};{"BIGINT":1003},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176654453553, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:1741661297536408, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f7ac86373d0, valid:true, source:2, core:{version:{val:1742176624548933000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176624553228, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7a81b2bbd0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7a81b2bbf0, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a81b2bb90, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7a81b2bc40, count:4, allocator:0x7f7adab4c6b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f7ac86339a0, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f7ac86376a0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7a81b28338}], op_filters:null, row2exprs_projector:0x7f7ac8633848, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f7ac8637ac0, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:57:04.556209] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=299][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:04.556215] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=6][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:57:04.556219] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:57:04.556223] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:57:04.556228] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:57:04.556231] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:04.556236] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7ac862fb50}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:04.556247] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=11][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:57:04.556254] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:57:04.556261] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=6][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:57:04.556268] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=6][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:57:04.556275] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f7ac862fb50}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:57:04.556283] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:04.556287] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=3][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:04.556298] WDIAG [SERVER] open (ob_inner_sql_result.cpp:167) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=10][errcode=-4016] get_next_row failed(ret=-4016, has_tenant_resource()=true) [2025-03-17 09:57:04.556309] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=10][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}) [2025-03-17 09:57:04.556321] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=11][errcode=-4016] execute failed(ret=-4016, tenant_id=1, executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741661297536408) [2025-03-17 09:57:04.556336] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=13][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:04.556367] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=13] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ace865ba0, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222524024, assoc_session_id:3222524024, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176624554292, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176624553228, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f797408af58, id:3222524024, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7ace865ba0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.556414] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=43][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:04.556419] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:04.556423] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:04.556443] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539363][T1003_COMMONLSS][T1][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=14][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:04.556451] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=6][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003"}, aret=-4016, ret=-4016) [2025-03-17 09:57:04.556457] WDIAG [SERVER] execute_read_inner (ob_inner_sql_connection.cpp:1806) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=5][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:57:04.556466] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=8][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1) [2025-03-17 09:57:04.556472] WDIAG [SERVER] execute_read (ob_inner_sql_connection.cpp:1746) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=5][errcode=-4016] execute_read failed(ret=-4016, cluster_id=1741272511, tenant_id=1) [2025-03-17 09:57:04.556480] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:138) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=7][errcode=-4016] query failed(ret=-4016, conn=0x7f7b317c70f0, start=1742176624554249, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:57:04.556489] WDIAG [COMMON.MYSQLP] read (ob_mysql_proxy.cpp:67) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=8][errcode=-4016] read failed(ret=-4016) [2025-03-17 09:57:04.556494] WDIAG [SHARE] exec_read (ob_ls_i_life_manager.h:213) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=4][errcode=-4016] failed to read(ret=-4016, ret="OB_ERR_UNEXPECTED", exec_tenant_id=1, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:57:04.556548] WDIAG [SHARE] get_ls_primary_zone_info (ob_ls_status_operator.cpp:1062) [3539363][T1003_COMMONLSS][T1003][YB420A0B093E-00062FB7FD3E0078-0-0] [lt=8][errcode=-4016] failed to read ls recovery(ret=-4016, ret="OB_ERR_UNEXPECTED", tenant_id=1003, sql=select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = 1 and a.tenant_id = 1003) [2025-03-17 09:57:04.564587] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=25] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:04.564639] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117783][T1_TimerWK2][T1][Y0-0000000000000000-0-0] [lt=25] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:04.566896] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463554440}], scn_array=[cnt:1, {val:1742176624548933000, v:0}], curr_size=587, accum_size=2004, time_cost=16921) [2025-03-17 09:57:04.577932] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=27][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.577982] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=49][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, offset=1594640) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.578003] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=20][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.578011] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9930, ref_cnt_:1, raw_buf_:0x7f7ac6b6c150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624577392, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:500, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:04.578064] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=43][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310c9930, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624577392, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:500, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:-1, end_used:1742176624578057}}, part_id:-1}) [2025-03-17 09:57:04.578074] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=0][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624577392, enqueue_used:2, dequeue_used:18, submit_used:1, return_used:500, callback_enqueue_used:2, callback_dequeue_used:12, callback_finish_used:169, end_used:-39}}) [2025-03-17 09:57:04.578112] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=36][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:04.578125] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=11][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:04.578459] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=32][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=295, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.578475] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=15][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:04.578483] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=8][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f7a41073510, src_buf_size=295) [2025-03-17 09:57:04.578491] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310ca428, ref_cnt_:1, raw_buf_:0x7f7a41072150, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624578136, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:275, callback_enqueue_used:1, callback_dequeue_used:21, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1) [2025-03-17 09:57:04.578519] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [117786][T1_DiskCB0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=19][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1, control_block_:0x7f7b310ca428, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606860212, second_id:13237, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B0940-00063023DBCBBE1F-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f7b311e4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624578136, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:275, callback_enqueue_used:1, callback_dequeue_used:21, callback_finish_used:-1, end_used:1742176624578511}}, part_id:-1}) [2025-03-17 09:57:04.578528] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1594640, size_:295, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1, tenant_io_mgr_:{ptr:0x7f7b311e4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a602dc080, time_log:{begin_ts:1742176624578136, enqueue_used:1, dequeue_used:23, submit_used:1, return_used:275, callback_enqueue_used:1, callback_dequeue_used:21, callback_finish_used:79, end_used:-26}}) [2025-03-17 09:57:04.578547] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=18][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=13236][2nd=13237]}, ret=-4016) [2025-03-17 09:57:04.578556] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=8][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a602dc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.578578] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=21][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, offset:1594640, size:295, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a602dc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.578600] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=21][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1, macro_block_id={[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info={offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:04.578618] WDIAG [STORAGE] lookup_in_index_tree (ob_index_tree_prefetcher.cpp:240) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=17][errcode=-4016] Fail to get index block data(ret=-4016, curr_handle={tenant_id:1, macro_block_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, micro_info:{offset:1594640, size:295, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}, is_loaded_block:false, block_state:2, block_index:-1, cache_handle:{handle:{mb_handle:null}, micro_block:null}, io_handle:{macro_id:{[ver=1,mode=0,seq=13236][2nd=13237]}, io_handle:{io_result:"NULL"}}, loaded_block_data:{buf:null, size:0, extra_buf:null, extra_size:0, type:0}, allocator:0x7f7b0cc52090}) [2025-03-17 09:57:04.578643] WDIAG [STORAGE] inner_get_next_row (ob_sstable_row_getter.cpp:117) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=24][errcode=-4016] Fail to prefetch(ret=-4016, read_handle={is_get:true, is_bf_contain:false, is_sorted_multi_get:false, row_state:3, range_idx:0, index_block_info:{query_range:0x7f7b0cc52e38, row_header:{version:2, row_store_type:0, compressor_type:1, is_data_index:1, is_data_block:0, is_leaf_block:1, is_major_node:0, is_pre_aggregated:0, is_deleted:0, contain_uncommitted_row:0, is_macro_node:1, has_string_out_row:0, all_lob_in_row:0, has_logic_micro_id:1, get_macro_id():{[ver=1,mode=0,seq=13236][2nd=13237]}, block_offset:1296, block_size:295, master_key_id:0, encrypt_id:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", row_count:5, schema_version:1741278087501752, macro_block_count:1, micro_block_count:1, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0, shared_data_macro_id:{[ver=0,mode=0,seq=0][2nd=0]}, has_shared_data_macro_id:0}, minor_meta_info:{snapshot_version:1741629604022598000, max_merged_trans_version:1741595943694436000, row_count_delta:1}, endkey:{type:2, key_ptr:0x7f7a8d883c68, }, ps_node:0x7f7a8d8839a0, agg_row_buf:null, agg_buf_size:0, flag:66, range_idx:0, parent_macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, nested_offset:1593344, rowkey_begin_idx:-1, rowkey_end_idx:1152921504606846976, cs_row_range:{start_row_id:-4, end_row_id:0}, skipping_filter_results:{size:0, ObArrayWrap(skipping_filter_results_, size_):[]}, table_read_info:null}, micro_begin_idx:-1, micro_end_idx:-1, query_range:0x7f7b0cc52e38, micro_handle:NULL}) [2025-03-17 09:57:04.578689] WDIAG [STORAGE] get_table_row (ob_single_merge.cpp:156) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=46][errcode=-4016] Fail to get row, (ret=-4016, table_idx=1) [2025-03-17 09:57:04.578696] WDIAG [STORAGE] get_and_fuse_cache_row (ob_single_merge.cpp:227) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=7][errcode=-4016] fail to get table row(ret=-4016, i=1, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f781dc14fa8, local_datums:0x7f781dc14fa8}, datums:[]}, tables_=[{ObITable:{this:0x7f7a8d882898, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MAJOR", scn_range:{start_scn:{val:0, v:0}, end_scn:{val:1741543202134877000, v:0}}}, ref_cnt:0, upper_trans_version:0, timestamp:0}, this:0x7f7a8d882898, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=299008,size=1319,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:0, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:659456, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:2, occupy_size:877, data_checksum:3826204589, max_merged_trans_version:1741315834549998001, upper_trans_version:0, filled_tx_scn:{val:0, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:2, occupy_size_:877, original_size_:278, data_checksum_:3826204589, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087142176, create_snapshot_version_:1741456800355942000, progressive_merge_round_:1, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741315834549998001, max_merged_trans_version_:1741315834549998001, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:0, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:2, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:15, 0:3854601585,1:6689739948,2:1109737249,3:4777684706,4:5988097670,5:4506807425,6:6116882180,7:6116882180,8:1764428918,9:4777684706,10:4777684706,11:4272592816,12:6116882180,13:6116882180,14:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=414,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d882cd8, block_data:{buf:0x7f7a8d882c70, size:104, extra_buf:0x7f7a8d882e76, extra_size:1007, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=463,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883265, block_data:{buf:0x7f7a8d883265, size:463, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883434, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:659456, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883464, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741370403584048000, v:0}, end_scn:{val:1741629604022598000, v:0}}}, ref_cnt:0, upper_trans_version:1741595943694436000, timestamp:0}, this:0x7f7a8d883464, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=300327,size=935,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:1593344, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741595943694436000, upper_trans_version:1741595943694436000, filled_tx_scn:{val:1741629604022598000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:114, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:252679375, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741595943694436000, max_merged_trans_version_:1741595943694436000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741629604022598000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883804, block_data:{buf:0x7f7a8d8837c4, size:64, extra_buf:0x7f7a8d88392b, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d883d32, block_data:{buf:0x7f7a8d883d32, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d883e38, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:1593344, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}, {ObITable:{this:0x7f7a8d883e58, key:{tablet_id:{id:345}, column_group_idx:0, table_type:"MINI", scn_range:{start_scn:{val:1741629604022598000, v:0}, end_scn:{val:1741728830844409000, v:0}}}, ref_cnt:0, upper_trans_version:1741661268240025000, timestamp:0}, this:0x7f7a8d883e58, addr:{[ver=1,mode=0,seq=20239][2nd=20240][3rd=0][offset=301262,size=936,type=2,seq=0][trans_seq=0, sec_part=0][6th=0]}, meta_cache:{version:1, has_multi_version_row:1, status:2, data_macro_block_count:1, nested_size:4096, nested_offset:704512, total_macro_block_count:1, total_use_old_macro_block_count:0, row_count:5, occupy_size:1296, data_checksum:0, max_merged_trans_version:1741661268240025000, upper_trans_version:1741661268240025000, filled_tx_scn:{val:1741728830844409000, v:0}, contain_uncommitted_row:false}, meta:{basic_meta:{version:1, length:115, row_count_:5, occupy_size_:1296, original_size_:927, data_checksum_:3171086899, index_type_:0, rowkey_column_count_:4, column_cnt_:15, data_macro_block_count_:1, data_micro_block_count_:1, use_old_macro_block_count_:0, index_macro_block_count_:0, sstable_format_version_:1, schema_version_:1741278087501752, create_snapshot_version_:0, progressive_merge_round_:0, progressive_merge_step_:0, data_index_tree_height_:2, table_mode_:{table_mode_flag:0, pk_mode:0, table_state_flag:0, view_created_method_flag:0, table_organization_mode:0, auto_increment_mode:0, rowid_mode:0, view_column_filled_flag:0, mv_container_table_flag:0, mv_available_flag:0, table_referenced_by_mv_flag:0, mv_enable_query_rewrite_flag:0, mv_on_query_computation_flag:0, ddl_table_ignore_sync_cdc_flag:0}, upper_trans_version_:1741661268240025000, max_merged_trans_version_:1741661268240025000, recycle_version:0, ddl_scn_:{val:0, v:0}, filled_tx_scn_:{val:1741728830844409000, v:0}, contain_uncommitted_row_:false, status_:4, root_row_store_type:0, compressor_type:1, encrypt_id:0, master_key_id:0, sstable_logic_seq:0, encrypt_key:"data_size:16, data:00000000000000000000000000000000", latest_row_store_type:1, table_backup_flag:{has_backup_flag:0, has_local_flag:0}, table_shared_flag:{shared_flag:0, reserved:0}, root_macro_seq:8796093022208, co_base_snapshot_version:0}, column_ckm_struct:{count:0}, data_root_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=295,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d8841f8, block_data:{buf:0x7f7a8d8841b8, size:64, extra_buf:0x7f7a8d88431f, extra_size:1031, type:1}}, macro_info:{macro_meta_info:{addr:{[ver=1,mode=0,seq=0][2nd=0][3rd=0][offset=0,size=262,type=3,seq=0][trans_seq=0, sec_part=0][6th=0]}, orig_block_buf:0x7f7a8d884726, block_data:{buf:0x7f7a8d884726, size:262, extra_buf:null, extra_size:0, type:1}}, data_block_count:1, other_block_count:0, linked_block_count:0, data_block_ids:0x7f7a8d88482c, other_block_ids:null, linked_block_ids:null, entry_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, is_meta_root:true, nested_offset:704512, nested_size:4096}, cg_sstables:{cnt:0, sstable_array:null, serialize_table_type:false, is_inited:true}, tx_ctx:{count:0, ObArrayWrap(tx_descs_, count_):null}, is_inited:true}, valid_for_reading:true, is_tmp_sstable:false}]) [2025-03-17 09:57:04.578832] WDIAG [STORAGE] get_normal_table_scan_row (ob_single_merge.cpp:367) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=135][errcode=-4016] Failed to get fuse cache row(ret=-4016, full_row_={row_flag:{flag:"NOT_EXIST", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:0, datum_buffer:{capacity:16, datums:0x7f781dc14fa8, local_datums:0x7f781dc14fa8}, datums:[]}) [2025-03-17 09:57:04.578842] WDIAG [STORAGE] inner_get_next_row (ob_single_merge.cpp:286) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=10][errcode=-4016] Failed to get normal row(ret=-4016, read_snapshot_version=1742176624548933000, tablet_meta.multi_version_start_=1741629603950087000, enable_fuse_row_cache=true) [2025-03-17 09:57:04.578848] WDIAG [STORAGE] get_next_row (ob_multiple_merge.cpp:434) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=5][errcode=-4016] Fail to inner get next row, (ret=-4016, this=0x7f781dc14010) [2025-03-17 09:57:04.578852] WDIAG [STORAGE] get_next_rows (ob_multiple_merge.cpp:510) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=4][errcode=-4016] failed to get single row(ret=-4016) [2025-03-17 09:57:04.578857] WDIAG [STORAGE] get_next_rows (ob_table_scan_iterator.cpp:723) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=5][errcode=-4016] Fail to get next row, (ret=-4016, *scan_param_={tablet_id:{id:345}, ls_id:{id:1}, column_ids:[18, 19, 25, 23], index_id:345, key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,[{"BIGINT":1001},{"BIGINT":1};{"BIGINT":1001},{"BIGINT":1}]"}], ss_key_ranges:[{"range":"table_id:345,group_idx:0,index_ordered_idx:0,(MIN;MAX)"}], range_array_pos:[], timeout:1742176654475581, scan_flag:{scan_order:1, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:0, prewarm:0, join_type:0, use_row_cache:0, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:1, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:4194304, reserved_cell_count:4, schema_version:1741278087142176, query_begin_schema_version:-1, limit_offset:{offset_:0, limit_:-1}, for_update:false, wait:-1, frozen_version:-1, is_get:true, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], op_filters:NULL, trans_desc:NULL, snapshot:{this:0x7f781dc13920, valid:true, source:2, core:{version:{val:1742176624548933000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}, table_param:{table_id:345, output_projector:[cnt:4, 0, 1, 2, 3], aggregate_projector:[cnt:0], group_by_projector:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], pad_col_projector:[cnt:0], read_param_version:1, main_read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7ad2ac3da0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7ad2ac3dc0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7ad2ac3d60, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7ad2ac3e10, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, use_lob_locator:false, rowid_version:0, rowid_projector:[cnt:0], enable_lob_locator_v2:true, is_fts_index:false, parser_name:"", is_vec_index:false, is_column_replica_table:false, is_normal_cgs_at_the_end:true}, sample_info:{method:0, percent:"1.000000000000000000e+02", seed:-1, table_id:0, scope:0, force_block:false}, need_scn:false, need_switch_param:false, is_mds_query:false, fb_read_tx_uncommitted:false, external_file_format:{"TYPE":"INVALID"}, external_file_location:"", tx_seq_base:1742176624576612, need_update_tablet_param:true}, main_table_param={iter_param:{table_id:345, tablet_id:{id:345}, cg_idx:0, read_info:{schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:11, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:4, 0, 1, 11, 9], data:0x7f7ad2ac3da0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:4, 0, 1, 9, 7], data:0x7f7ad2ac3dc0, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=23 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7ad2ac3d60, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:4, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}, {pack:0, min_max:0, sum:0}], data:0x7f7ad2ac3e10, count:4, allocator:0x7f7a902c86b0, capacity:4, init_cnt:4}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, rowkey_read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:13, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:15}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=19 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f7a8d882668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:15}}, out_cols_project:[cnt:4, 0, 1, 2, 3], agg_cols_project:[cnt:0], group_by_cols_project:[cnt:0], pushdown_filter:NULL, op:0x7f781dc0ff50, sstable_index_filter:null, cg_read_infos:null, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], aggregate_exprs:[cnt:0], output_sel_mask:[cnt:4, true, true, true, true], is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:true, pd_storage_flag:{pd_flag:16}, vectorized_enabled:true, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:true, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:8192}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:[cnt:0], is_tablet_spliting:false, need_update_tablet_param:0x7f781dc13bf0}, padding_cols:[cnt:0], projector_size:0, output_exprs:[cnt:4, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:0, res_buf_off:4071, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3471, pvt_skip_off:192, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2592, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230080}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:1, frame_idx:1, datum_off:200, res_buf_off:4199, dyn_buf_header_offset:3559, res_buf_len:8, eval_flags_off:3479, pvt_skip_off:392, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2657, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230168}, {type:"T_REF_COLUMN", datum_meta:{type:"VARCHAR", cs_type:"utf8mb4_general_ci", scale:-1, precision:-1}, obj_meta:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"IMPLICIT"}, obj_datum_map:1, flag:131, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:400, res_buf_off:4327, dyn_buf_header_offset:3559, res_buf_len:128, eval_flags_off:3487, pvt_skip_off:592, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2722, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230250}, {type:"T_REF_COLUMN", datum_meta:{type:"BIGINT", cs_type:"binary", scale:0, precision:20}, obj_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, obj_datum_map:3, flag:67, eval_func:null, eval_batch_func:null, inner_functions:null, inner_func_cnt:0, arg_cnt:0, parent_cnt:0, frame_idx:1, datum_off:600, res_buf_off:6375, dyn_buf_header_offset:3815, res_buf_len:8, eval_flags_off:3495, pvt_skip_off:792, expr_ctx_id:4294967295, extra:0, batch_idx_mask:18446744073709551615, vector_header_off:2787, eval_vector_func:null, local_session_var_id:-1, this:0x7f7aa3230338}], op_filters:null, row2exprs_projector:0x7f781dc0fdf8, output_sel_mask:[cnt:4, true, true, true, true], is_inited:true}, single_merge_=0x7f781dc14010, get_merge_=NULL, scan_merge_=NULL, multi_scan_merge_=NULL, skip_scan_merge_=NULL, cached_iter_node_=NULL) [2025-03-17 09:57:04.579113] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_scan_iter.cpp:130) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=255][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:04.579121] WDIAG [SQL.DAS] get_next_seq_rows (ob_das_merge_iter.cpp:592) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=8][errcode=-4016] das iter failed to get next rows(ret=-4016) [2025-03-17 09:57:04.579128] WDIAG [SQL.DAS] inner_get_next_rows (ob_das_merge_iter.cpp:386) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=6][errcode=-4016] das merge iter failed to get next rows(ret=-4016) [2025-03-17 09:57:04.579137] WDIAG [SQL.ENG] get_next_batch_with_das (ob_table_scan_op.cpp:2412) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=8][errcode=-4016] get next batch from das result failed(ret=-4016) [2025-03-17 09:57:04.579158] WDIAG [SQL.ENG] inner_get_next_batch_for_tsc (ob_table_scan_op.cpp:2605) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=20][errcode=-4016] get next batch with mode failed(ret=-4016) [2025-03-17 09:57:04.579163] WDIAG [SQL.ENG] inner_get_next_batch (ob_table_scan_op.cpp:2549) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=4][errcode=-4016] failed to get next batch(ret=-4016) [2025-03-17 09:57:04.579167] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=3][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f781dc0c100}, id=1, op_name="PHY_VEC_TABLE_SCAN") [2025-03-17 09:57:04.579177] WDIAG [SQL.ENG] get_next_left_batch (ob_group_join_buffer_v2.cpp:514) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=10][errcode=-4016] failed to get batch from left child(ret=-4016) [2025-03-17 09:57:04.579188] WDIAG [SQL.ENG] get_next_left_row (ob_group_join_buffer_v2.cpp:571) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=10][errcode=-4016] failed to get next left batch(ret=-4016) [2025-03-17 09:57:04.579195] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=6][errcode=-4016] failed to get next left row from friver iterator(ret=-4016) [2025-03-17 09:57:04.579201] WDIAG [SQL.ENG] inner_get_next_batch (ob_nested_loop_join_vec_op.cpp:385) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=5][errcode=-4016] fail to get left batch(ret=-4016) [2025-03-17 09:57:04.579208] WDIAG [SQL.ENG] get_next_batch (ob_operator.cpp:1387) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=6][errcode=-4016] get next batch failed(ret=-4016, eval_ctx={batch_idx:0, batch_size:16, max_batch_size:16, frames_:0x7f781dc0c100}, id=0, op_name="PHY_VEC_NESTED_LOOP_JOIN") [2025-03-17 09:57:04.579216] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1896) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=7][errcode=-4016] get next batch failed(ret=-4016) [2025-03-17 09:57:04.579223] WDIAG [SQL.EXE] sync_send_result (ob_remote_executor_processor.cpp:286) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=6][errcode=-4016] failed to get next row(ret=-4016) [2025-03-17 09:57:04.579230] WDIAG [SQL.EXE] execute_remote_plan (ob_remote_executor_processor.cpp:485) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=6][errcode=-4016] sync send result failed(ret=-4016) [2025-03-17 09:57:04.579246] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=7] end stmt(ret=0, tx_id=0, plain_select=true, stmt_type=1, savepoint=0, tx_desc={this:0x7f7ae6ae1280, tx_id:{txid:0}, state:1, addr:"192.168.1.62:2882", tenant_id:1, session_id:3222519956, assoc_session_id:3222519956, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:-1, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:0, op_sn:1, alloc_ts:1742176624576613, active_ts:-1, commit_ts:-1, finish_ts:-1, timeout_us:-1, lock_timeout_us:-1, expire_ts:9223372036854775807, coord_id:{id:-1}, parts:[], exec_info_reap_ts:0, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:-1, cluster_version:17180067072, seq_base:1742176624576612, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:false, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:1}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f788864e1f8, id:3222519956, deser:true, tenant:"", tenant_id:1, effective_tenant:"", effective_tenant_id:1, database:"oceanbase", user:"root@%", consistency_level:3, session_state:2, autocommit:true, tx:0x7f7ae6ae1280}, exec_ctx.get_errcode()=0) [2025-03-17 09:57:04.579288] WDIAG [SQL.EXE] execute_with_sql (ob_remote_executor_processor.cpp:716) [117882][T1_L5_G0][T1][YB420A0B0940-00063023DBCBBE1F-0-0] [lt=37][errcode=-4016] execute remote plan failed(ret=-4016, task={tenant_schema_version:1741661297536408, sys_schema_version:1741661297536408, runner_svr:"192.168.1.62:2882", ctrl_svr:"192.168.1.64:2882", task_id:{ob_job_id:{ob_execution_id:{server:"192.168.1.64:2882", execution_id:18446744073709551615, task_type:0, hash:4219408603045787357}, job_id:18446744073709551615}, task_id:0, task_cnt:0}, remote_sql_info:{use_ps:true, is_batched_stmt:false, is_original_ps_mode:false, ps_param_cnt:2, remote_sql:"select a.tenant_id as tenant_id, a.ls_id as ls_id, a.primary_zone as primary_zone, a.ls_group_id as ls_group_id, b.zone_priority as zone_priority from __all_ls_status as a left join __all_ls_election_reference_info b on a.tenant_id = b.tenant_id and a.ls_id = b.ls_id where a.ls_id = ? and a.tenant_id = ?", ps_params:[{obj:{"BIGINT":1}, accuracy:{length:1, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}, {obj:{"BIGINT":1001}, accuracy:{length:4, precision:20, scale:0}, flag:1, raw_text_pos:-1, raw_text_len:-1, param_meta:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}}]}, snapshot:{this:0x7f781dc04208, valid:false, source:0, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}}, exec_ctx_.get_das_ctx().get_snapshot()={this:0x7f781dc094a0, valid:true, source:2, core:{version:{val:1742176624548933000, v:0}, tx_id:{txid:0}, scn:0}, uncertain_bound:0, snapshot_lsid:{id:1}, snapshot_ls_role:1, snapshot_acquire_addr:"192.168.1.62:2882", parts:[], committed:false}) [2025-03-17 09:57:04.583911] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.605759] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=11] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:10, job_name_:"SPM_STATS_MANAGER", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742058000001265, start_date_:1741626000000000, last_date_:1741971600000972, this_date_:1742058000001265, next_date_:1742144400000000, end_date_:64060560000000000, total_:134405, interval_:"FREQ=DAYLY; INTERVAL=1", repeat_interval_:"FREQ=DAYLY; INTERVAL=1", failures_:5, flag_:0, what_:"DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:86400000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:04.607443] INFO [COMMON] handle (ob_io_struct.cpp:3072) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=1] callback runner call handle(get_queue_count()=0, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310cb430, ref_cnt_:1, raw_buf_:0x7f7aa4262150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63414-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624606944, enqueue_used:1, dequeue_used:21, submit_used:1, return_used:417, callback_enqueue_used:3, callback_dequeue_used:55, callback_finish_used:-1, end_used:-1}}, part_id:-1}) [2025-03-17 09:57:04.607497] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=52][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.607520] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=22][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.607541] WDIAG [STORAGE] process (ob_micro_block_cache.cpp:620) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=21][errcode=-4016] process_block failed(ret=-4016) [2025-03-17 09:57:04.607549] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310cb430, ref_cnt_:1, raw_buf_:0x7f7aa4262150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63414-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624606944, enqueue_used:1, dequeue_used:21, submit_used:1, return_used:417, callback_enqueue_used:3, callback_dequeue_used:55, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.607576] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=20][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310cb430, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63414-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624606944, enqueue_used:1, dequeue_used:21, submit_used:1, return_used:417, callback_enqueue_used:3, callback_dequeue_used:55, callback_finish_used:-1, end_used:1742176624607572}}, part_id:-1}) [2025-03-17 09:57:04.607582] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=38][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624606944, enqueue_used:1, dequeue_used:21, submit_used:1, return_used:417, callback_enqueue_used:3, callback_dequeue_used:55, callback_finish_used:153, end_used:-23}}) [2025-03-17 09:57:04.607619] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=35][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:04.607631] WDIAG [STORAGE] get_loaded_block_data (ob_micro_block_handle_mgr.cpp:205) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=10][errcode=-4016] Fail to wait micro block io(ret=-4016) [2025-03-17 09:57:04.607868] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=19][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.607896] WDIAG [STORAGE] deserialize_and_check_header (ob_micro_block_header.cpp:174) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=28][errcode=-4016] fail to deserialize header(ret=-4016) [2025-03-17 09:57:04.607909] WDIAG [STORAGE] inner_process (ob_micro_block_cache.cpp:858) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=13][errcode=-4016] fail to deserialize_and_check_header(ret=-4016, src_block_buf=0x7f79957b915a, src_buf_size=1611) [2025-03-17 09:57:04.607917] WDIAG [COMMON] handle (ob_io_struct.cpp:3100) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] fail to callback(ret=-4016, *req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c86a0, ref_cnt_:1, raw_buf_:0x7f79957b8150, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63414-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:false, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:0, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624607642, enqueue_used:2, dequeue_used:27, submit_used:0, return_used:185, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:-1}}, part_id:-1}, MTL_ID()=1003) [2025-03-17 09:57:04.607950] WDIAG [COMMON] run1 (ob_io_struct.cpp:2985) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=22][errcode=-4016] handle request failed(ret=-4016, req={is_inited_:true, tenant_id_:1003, control_block_:0x7f7b310c86a0, ref_cnt_:1, raw_buf_:null, fd_:{first_id:1152921504606882095, second_id:35120, third_id:0, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333b130}, trace_id_:YB420A0B093E-00062FB95BF63414-0-0, retry_count_:0, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, storage_accesser:{ptr:null}, io_result_:{is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:2, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624607642, enqueue_used:2, dequeue_used:27, submit_used:0, return_used:185, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:-1, end_used:1742176624607942}}, part_id:-1}) [2025-03-17 09:57:04.607974] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] IO error, (ret=-4016, *result_={is_inited_:true, is_finished_:true, is_canceled_:false, has_estimated_:false, complete_size_:4096, offset_:1581402, size_:1611, timeout_us_:10000000, result_ref_cnt_:1, out_ref_cnt_:1, flag_:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, ret_code_:{io_ret_:-4016, fs_errno_:0}, tenant_id_:1003, tenant_io_mgr_:{ptr:0x7f79ab3f4030}, user_data_buf_:null, buf_:null, io_callback_:0x7f7a0e5bc080, time_log:{begin_ts:1742176624607642, enqueue_used:2, dequeue_used:27, submit_used:0, return_used:185, callback_enqueue_used:1, callback_dequeue_used:9, callback_finish_used:102, end_used:-26}}) [2025-03-17 09:57:04.608035] WDIAG [STORAGE] wait (ob_storage_object_handle.cpp:378) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=60][errcode=-4016] fail to wait block io, may be retry(macro_id_={[ver=1,mode=0,seq=35119][2nd=35120]}, ret=-4016) [2025-03-17 09:57:04.608044] WDIAG [STORAGE] read_object (ob_object_manager.cpp:365) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] Fail to wait io finish(ret=-4016, read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.608063] WDIAG [STORAGE] load_block (ob_micro_block_cache.cpp:1576) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=19][errcode=-4016] Fail to sync read block(ret=-4016, macro_read_info={macro_block_id:{[ver=1,mode=0,seq=35119][2nd=35120]}, offset:1581402, size:1611, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, micro_crc:0, io_timeout_ms:10000, io_desc:{mode:"READ", group_id_:0, func_type_:0, wait_event_id_:1, is_sync_:false, is_unlimited_:false, is_detect_:false, is_write_through_:false, is_sealed_:true, is_time_detect_:false, need_close_dev_and_fd_:false, reserved_:0}, io_callback:0x7f7a0e5bc080, buf:null, ls_epoch_id:0, mtl_tenant_id:1003, bypass_micro_cache:false, is_major_macro_preread:false}) [2025-03-17 09:57:04.608125] WDIAG [STORAGE] get_micro_block_data (ob_micro_block_handle_mgr.cpp:125) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=59][errcode=-4016] Fail to load micro block(ret=-4016, tenant_id=1003, macro_block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, micro_info={offset:1581402, size:1611, mark_deletion:false, logic_micro_id:{version:0, offset:0, logic_macro_id:{data_seq:{data_seq:0, parallel_idx:0, block_type:0, merge_type:0, sstable_logic_seq:0, reserved:0, sign:0, macro_data_seq:0}, logic_version:0, tablet_id:0, column_group_idx:0, is_mds:false}}, data_checksum:0}) [2025-03-17 09:57:04.608163] WDIAG [STORAGE] forward (ob_index_tree_prefetcher.cpp:1889) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=37][errcode=-4016] Fail to get index block data(ret=-4016, this={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:04.608189] WDIAG [STORAGE] drill_down (ob_index_tree_prefetcher.cpp:1375) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=26][errcode=-4016] fail to consume tree handle(ret=-4016, cur_level_=0, tree_handles_[cur_level_ + 1]={is_prefetch_end:false, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:0, raw_iter:null, transformed_iter:null, ddl_iter:null, ddl_merge_iter:null, iter:NULL, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:false, is_right_border:false, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=9223372036854775807]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:-1}, {end_prefetched_row_idx:-1}]}, tree_handles_[cur_level_]={is_prefetch_end:true, can_blockscan:false, fetch_idx:0, prefetch_idx:0, read_idx:0, index_scanner:{index_format:2, raw_iter:null, transformed_iter:0x7f798ff46168, ddl_iter:null, ddl_merge_iter:null, iter:{base iterator::{base iterator::{is_inited_:true, is_reverse_scan_:false, iter_step_:1, datum_utils_:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}}, format::"ObRAWIndexBlockRowIterator", current_:1, start_:0, end_:0, micro_reader_:null, endkey_:{datum_cnt:0, group_idx:0, hash:0, [{}], store_rowkey:}, datum_row:null, allocator:null}, format::"ObTFMIndexBlockRowIterator", idx_data_header_:{row_cnt:1, col_cnt:5, rowkey_vector:{col_cnt:4, row_cnt:1, is_datum_vectors:-1, columns:0x7f798885fb28, last_rowkey:0x7f798885fca8, discrete_rowkey_array:0x7f798885fdc8, last_rowkey_:{datum_cnt:4, group_idx:0, hash:0, [idx=0:{len: 16, flag: 0, null: 0, ptr: 0x7f798885fbe0, hex: 5745444E45534441595F57494E444F57, cstr: WEDNESDAY_WINDOW}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc28, hex: 94A7449F25300600, int: 1741788004263828}, idx=2:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc68, hex: 70AE44DB09EDD3E7, int: -1741788004273050000}, idx=3:{len: 8, flag: 0, null: 0, ptr: 0x7f798885fc80, hex: 0000000000000000, int: 0}], store_rowkey:}}, index_datum_array:0x7f798885faaf, ps_node_array:0x7f798885fae8, data_buf:0x7f798885faaf, data_buf_size:1055}}, range_idx:0, is_get:false, is_reverse_scan:false, is_left_border:true, is_right_border:true, rowkey_begin_idx:0, rowkey_end_idx:0, is_inited:true, macro_id:{[ver=1,mode=0,seq=0][2nd=18446744073709551615]}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, is_normal_cg:false, parent_row_range:{start_row_id:-1, end_row_id:-1}, filter_constant_type:0, is_normal_query:true, iter_param:{sstable_:0x7f798885f59b, tablet_:0x7f798885e0a0}, table_read_info:null}, ObArrayWrap(index_block_read_handles_, INDEX_TREE_PREFETCH_DEPTH):[{end_prefetched_row_idx:0}, {end_prefetched_row_idx:-1}]}) [2025-03-17 09:57:04.608232] WDIAG [STORAGE] prefetch_micro_data (ob_index_tree_prefetcher.cpp:1274) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=42][errcode=-4016] fail to get index leaf(ret=-4016, cur_level_=0, prefetch_depth=1, prefetched_cnt=0) [2025-03-17 09:57:04.608237] WDIAG [STORAGE] prefetch (ob_index_tree_prefetcher.cpp:1138) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] Fail to prefetch(ret=-4016, index_tree_height_=2, cur_level_=0) [2025-03-17 09:57:04.608243] WDIAG [STORAGE] inner_open (ob_sstable_row_scanner.cpp:150) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] ObSSTableRowScanner prefetch failed(ret=-4016) [2025-03-17 09:57:04.608251] WDIAG [STORAGE] inner_open (ob_sstable_row_lock_checker.cpp:56) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] failed to open scanner(ret=-4016) [2025-03-17 09:57:04.608255] WDIAG [STORAGE] init (ob_store_row_iterator.cpp:64) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] Failed to inner open ObStoreRowIterator(ret=-4016, iter_param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, access_ctx={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176654506813, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}) [2025-03-17 09:57:04.608324] WDIAG [STORAGE] check_row_locked (ob_sstable.cpp:1006) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=69][errcode=-4016] failed to open row locker(ret=-4016, param={table_id:519, tablet_id:{id:519}, cg_idx:3101506029, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, context={is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176654506813, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}, rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294491c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 657AD81A80300600, int: 1742176624605797}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}}) [2025-03-17 09:57:04.608388] WDIAG [STORAGE.TRANS] internal_lock_row_on_frozen_stores_ (ob_memtable.cpp:1429) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=62][errcode=-4016] sstable check row lock fail(ret=-4016, key={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}, check_exist=true, datum_rowkey={datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294491c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f798ff424e0, hex: 657AD81A80300600, int: 1742176624605797}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}}, lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, tmp_lock_state={is_locked:false, trans_version:{val:0, v:0}, lock_trans_id:{txid:0}, lock_data_sequence:0, lock_dml_flag:0, is_delayed_cleanout:false, exist_flag:1, mvcc_row:null, trans_scn:{val:4611686018427387903, v:0}}, row_state={lock_is_decided:false, max_trans_version:{val:0, v:0}, row_dml_flag:0}) [2025-03-17 09:57:04.608410] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_ (ob_memtable.cpp:1335) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=21][errcode=-4016] internal lock row failed(ret=-4016, ret="OB_ERR_UNEXPECTED") [2025-03-17 09:57:04.608417] WDIAG [STORAGE.TRANS] lock_row_on_frozen_stores_on_failure (ob_memtable.cpp:1620) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] Failed to lock row on frozen store(ret=-4016) [2025-03-17 09:57:04.608422] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2736) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] mvcc write fail(memtable_key_generator.get_memtable_key()={"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}, ret=-4016) [2025-03-17 09:57:04.608460] WDIAG [STORAGE.TRANS] set_ (ob_memtable.cpp:2774) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=13][errcode=-4016] set end, fail(ret=-4016, tablet_id_={id:519}, columns="[column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0,column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0,]", new_row={row_flag:{flag:"INSERT", flag_type:0}, trans_id:{txid:0}, scan_index:0, mvcc_row_flag:{first:0, uncommitted:0, shadow:0, compact:0, ghost:0, last:0, reserved:0, flag:0}, snapshot_version:0, fast_filter_skipped:false, have_uncommited_row:false, group_idx:0, count:29, datum_buffer:{capacity:29, datums:0x7f7a22944e50, local_datums:0x7f7a22944aa0}, datums:[col_id=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294491c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, col_id=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2294492d, hex: 657AD81A80300600, int: 1742176624605797}, col_id=2:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944935, hex: 657AD81A80300600, int: 1742176624605797}, col_id=3:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2294493d, hex: 657AD81A80300600, int: 1742176624605797}, col_id=4:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944945, hex: 0A00000000000000, int: 10}, col_id=5:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2294494d, hex: 0000000000000000, int: 0}, col_id=6:null, col_id=7:{len: 6, flag: 0, null: 0, ptr: 0x7f7a22944955, hex: 726F6F744025, cstr: root@%}, col_id=8:null, col_id=9:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294495b, hex: 44454641554C545F4A4F425F434C415353, cstr: DEFAULT_JOB_CLASS}, col_id=10:{len: 36, flag: 0, null: 0, ptr: 0x7f7a2294496c, hex: 44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829, cstr: DBMS_SPM.HANDLE_SPM_STATS_JOB_PROC()}, col_id=11:{len: 9, flag: 0, null: 0, ptr: 0x7f7a22944990, hex: 5343484544554C4544, cstr: SCHEDULED}, col_id=12:{len: 8, flag: 0, null: 0, ptr: 0x7f7a22944999, hex: 54F0FFFFFFFFFFFF, int: -4012}, col_id=13:{len: 8, flag: 0, null: 0, ptr: 0x7f7a229449a1, hex: 00C40FE7FF2F0600, int: 1741626000000000}, col_id=14:null, col_id=15:null, col_id=16:null, col_id=17:null, col_id=18:null, col_id=19:null, col_id=20:null, col_id=21:null, col_id=22:null, col_id=23:null, col_id=24:null, col_id=25:null, col_id=26:null, col_id=27:{len: 17, flag: 0, null: 0, ptr: 0x7f7a229449a9, hex: 636865636B206A6F622074696D656F7574, cstr: check job timeout}, col_id=28:{len: 9, flag: 0, null: 0, ptr: 0x7f7a229449ba, hex: 6F6365616E62617365, cstr: oceanbase}]}, mem_ctx=0x7f79a5f0aed0, store_ctx={this:0x7f798ff48938, ls_id:{id:1}, ls:0x7f79a3dae150, branch:0, timeout:1742176654506813, tablet_id:{id:519}, table_iter:0x7f798ff47720, table_version:1741595950668896, mvcc_acc_ctx:{type:3, abs_lock_timeout_ts:1742176654506813, tx_lock_timeout_us:-1, snapshot:{version:{val:1742176624535125000, v:0}, tx_id:{txid:9634438}, scn:{branch:0, seq:4}}, tx_table_guards:{tx_table_guard:{tx_table:0x7f79a3db9b10, epoch:1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_tx_table_guard:{tx_table:null, epoch:-1, mini_cache_:[0:{}, 1:{}, 2:{}, 3:{}]}, src_ls_handle:{ls_map_:null, ls_:null, mod_:-1}}, tx_id:{txid:9634438}, tx_desc:{this:0x7f7acf2a26e0, tx_id:{txid:9634438}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222524219, assoc_session_id:3222524219, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176624605533, active_ts:1742176624605533, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654605527, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468721106949, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:3}, last_touch_ts:1742176624603417}], exec_info_reap_ts:1742176624603416, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624603413, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, tx_ctx:0x7f79a5f09950, mem_ctx:0x7f79a5f0aed0, tx_scn:{branch:0, seq:5}, write_flag:{is_table_api:0, is_table_lock:0, is_mds:0, is_dml_batch_opt:0, is_insert_up:0, is_write_only_index:0, is_check_row_locked:0, is_lob_aux:0, is_skip_flush_redo:0, is_update_uk:0, is_update_pk_dop:0}, handle_start_time:-1, lock_wait_start_ts:0}, tablet_stat:{ls_id:0, tablet_id:0, query_cnt:0, merge_cnt:0, scan_logical_row_cnt:0, scan_physical_row_cnt:0, scan_micro_block_cnt:0, pushdown_micro_block_cnt:0, exist_row_total_table_cnt:0, exist_row_read_table_cnt:0, insert_row_cnt:0, update_row_cnt:0, delete_row_cnt:0}, is_read_store_ctx:false, update_full_column:false}) [2025-03-17 09:57:04.608542] WDIAG [STORAGE] insert_rows (ob_tablet.cpp:4597) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=81][errcode=-4016] Failed to multi-set memtable(ret=-4016, row_count=1) [2025-03-17 09:57:04.608549] WDIAG [STORAGE] insert_tablet_rows (ob_ls_tablet_service.cpp:4523) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] Failed to insert rows to tablet(ret=-4016, rows_info={rowkeys:[{marked_rowkey:{row_mark:0, rowkey:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294491c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2294492d, hex: 657AD81A80300600, int: 1742176624605797}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}}}, lock_state:null, row_idx:0}], permutation:[0], min_key:{datum_cnt:2, group_idx:0, hash:0, [idx=0:{len: 17, flag: 0, null: 0, ptr: 0x7f7a2294491c, hex: 53504D5F53544154535F4D414E41474552, cstr: SPM_STATS_MANAGER}, idx=1:{len: 8, flag: 0, null: 0, ptr: 0x7f7a2294492d, hex: 657AD81A80300600, int: 1742176624605797}], store_rowkey:{"VARCHAR":"SPM_STATS_MANAGER", collation:"utf8mb4_general_ci"},{"TIMESTAMP":"2025-03-17 01:57:04.605797"}}, delete_count:0, conflict_rowkey_idx:-1, error_code:0, exist_helper:{table_iter_param:{table_id:519, tablet_id:{id:519}, cg_idx:0, read_info:{is_inited:true, compat_version:3, is_oracle_mode:false, is_cs_replica_compat:0, schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, cols_index:{rowkey_mode:true, for_memtable:false, schema_rowkey_cnt:2, column_cnt:31}, cols_desc:{ObIArray:[cnt:4, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=7 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=8 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0], data:0x7f798885e668, count:4, allocator:null, capacity:4, init_cnt:4}, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, memtable_cols_index:{rowkey_mode:true, for_memtable:true, schema_rowkey_cnt:2, column_cnt:31}}, rowkey_read_info:NULL, out_cols_project:NULL, agg_cols_project:NULL, group_by_cols_project:NULL, pushdown_filter:NULL, op:null, sstable_index_filter:null, cg_read_infos:null, output_exprs:NULL, aggregate_exprs:NULL, output_sel_mask:NULL, is_multi_version_minor_merge:false, need_scn:false, is_same_schema_column:false, pd_storage_flag:{pd_flag:0}, vectorized_enabled:false, has_virtual_columns:false, has_lob_column_out:false, is_for_foreign_check:false, limit_prefetch:false, is_mds_query:false, is_non_unique_local_index:false, ss_rowkey_prefix_cnt:0, table_scan_opt:{io_read_batch_size:0, io_read_gap_size:0, storage_rowsets_size:1}, auto_split_filter_type:18446744073709551615, auto_split_filter:null, auto_split_params:NULL, is_tablet_spliting:false, need_update_tablet_param:null}, table_access_context:{is_inited:true, use_fuse_row_cache:false, need_scn:false, need_release_mview_scan_info:false, timeout:1742176654506813, ls_id:{id:1}, tablet_id:{id:519}, query_flag:{scan_order:0, daily_merge:0, rmmb_optimize:0, whole_macro_scan:0, full_row:0, index_back:0, query_stat:0, sql_mode:0, read_latest:1, prewarm:0, join_type:0, use_row_cache:1, use_block_index_cache:0, use_bloomfilter_cache:0, multi_version_minor_merge:0, is_need_feedback:0, use_fuse_row_cache:0, use_fast_agg:0, iter_uncommitted_row:0, ignore_trans_stat:0, is_large_query:0, is_sstable_cut:0, skip_read_lob:0, is_lookup_for_4377:0, is_bare_row_scan:0, enable_rich_format:0, is_for_foreign_key_check:0, is_mds_query:0, is_select_follower:0, enable_lob_prefetch:0, is_bare_row_scan:0, mr_mv_scan:0, reserved:0}, sql_mode:0, store_ctx:0x7f798ff48938, limit_param:null, stmt_allocator:0x7f798ff45fe8, allocator:0x7f798ff45b40, range_allocator:null, table_scan_stat:null, out_cnt:0, trans_version_range:{multi_version_start:0, base_version:0, snapshot_version:4611686018427387902}, merge_scn:{val:4611686018427387903, v:0}, lob_allocator:pages=0, used=0, total=0, lob_locator_helper:null, cached_iter_node:null, stmt_iter_pool:null, cg_iter_pool:null, cg_param_pool:null, block_row_store:null, sample_filter:null, mview_scan_info:NULL}}}) [2025-03-17 09:57:04.608599] WDIAG [STORAGE] insert_rows_to_tablet (ob_ls_tablet_service.cpp:4455) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=49][errcode=-4016] failed to insert rows to data tablet(ret=-4016) [2025-03-17 09:57:04.608603] WDIAG [STORAGE] insert_rows (ob_ls_tablet_service.cpp:3185) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] insert to each tablets fail(ret=-4016) [2025-03-17 09:57:04.608611] WDIAG [SQL.DAS] write_rows (ob_das_insert_op.cpp:88) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=4][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:04.608619] WDIAG [SQL.DAS] write_tablet (ob_dml_service.h:355) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] write rows failed(ret=-4016, ls_id_={id:1}, tablet_id_={id:519}, ctdef_->table_id_=18446744073709551614, ctdef_->index_tid_=519) [2025-03-17 09:57:04.608632] WDIAG [SQL.DAS] insert_rows (ob_das_insert_op.cpp:139) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=9][errcode=-4016] insert rows to access service failed(ret=-4016) [2025-03-17 09:57:04.608638] WDIAG [SQL.DAS] open_op (ob_das_insert_op.cpp:114) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=6][errcode=-4016] fail to do insert(ret=-4016) [2025-03-17 09:57:04.608646] WDIAG [SQL.DAS] start_das_task (ob_das_task.cpp:349) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] open das task op failed(ret=-4016) [2025-03-17 09:57:04.608654] WDIAG [SQL.DAS] do_local_das_task (ob_data_access_service.cpp:382) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] start local das task failed(ret=-4016) [2025-03-17 09:57:04.608661] WDIAG [SQL.DAS] execute_dist_das_task (ob_data_access_service.cpp:184) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] do local das task failed(ret=-4016, task_arg={timeout_ts:1742176654606813, ctrl_svr:"192.168.1.62:2882", runner_svr:"192.168.1.62:2882", task_ops:[{parent:{tenant_id:1003, task_id:56389342, op_type:2, errcode:-4016, can_part_retry:0, task_started:1, in_part_retry:0, in_stmt_retry:0, need_switch_param:0, trans_desc:{this:0x7f7acf2a26e0, tx_id:{txid:9634438}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222524219, assoc_session_id:3222524219, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:4, alloc_ts:1742176624605533, active_ts:1742176624605533, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654605527, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468721106949, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:5}, last_touch_ts:1742176624603417}], exec_info_reap_ts:1742176624603416, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624603413, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, snapshot:{this:0x7f798ff4ee78, valid:true, source:5, core:{version:{val:18446744073709551615, v:3}, tx_id:{txid:9634438}, scn:{branch:0, seq:4}}, uncertain_bound:0, snapshot_lsid:{id:-1}, snapshot_ls_role:0, snapshot_acquire_addr:"0.0.0.0:0", parts:[], committed:false}, tablet_id:{id:519}, ls_id:{id:1}, tablet_loc:{tablet_id:{id:519}, ls_id:{id:1}, server:"192.168.1.62:2882", loc_meta:{table_loc_id:18446744073709551614, ref_table_id:519, related_table_ids:[cnt:2, 101105, 101106], use_dist_das:0, select_leader:1, is_dup_table:0, is_weak_read:0, unuse_related_pruning:0, is_external_table:0, route_policy:0}, in_retry:0, partition_id:18446744073709551615, first_level_part_id:18446744073709551615}, related_ctdefs:[cnt:2, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101105, schema_version:1741595950669840, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:2, 19, 18], column_types:[cnt:2, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:2, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:2, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101105, schema_version:1741595950669840, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_time", fts_parser_name:"", pk_name:"", columns:[cnt:2, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:2, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:1, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2920, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:2, 0, 1], data:0x7f77adbc2930, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}}, cols_desc:{ObIArray:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc2900, count:2, allocator:0x7f79d89f86b0, capacity:2, init_cnt:2}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:2, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, {ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:101106, schema_version:1741595950670232, rowkey_cnt:3, spk_cnt:0, column_ids:[cnt:3, 25, 19, 18], column_types:[cnt:3, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:3, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:3, 9, 1, 0], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:101106, schema_version:1741595950670232, table_type:5, index_type:1, index_status:2, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"idx_scheduler_job_run_detail_v2_job_class_time", fts_parser_name:"", pk_name:"", columns:[cnt:3, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:3, schema_rowkey_cnt:3, rowkey_cnt:5, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:3, max_col_index:2, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3420, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:3, 0, 1, 2], data:0x7f77adbc3430, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}}, cols_desc:{ObIArray:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77adbc33f0, count:3, allocator:0x7f79d89f86b0, capacity:3, init_cnt:3}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:5, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:3, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}], related_rtdefs:[cnt:2, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176654506813, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, {ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176654506813, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}], task_status:1, related_tablet_ids:[cnt:2, {id:101105}, {id:101106}], das_task_node:, plan_line_id:0}, ins_ctdef:{ObDASBaseCtDef:{ObDASBaseDef:{op_type:2, children_cnt:0}, table_id:18446744073709551614, index_tid:519, schema_version:1741595950668896, rowkey_cnt:2, spk_cnt:0, column_ids:[cnt:29, 18, 19, 16, 17, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44], column_types:[cnt:29, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}], column_accuracys:[cnt:29, {length:128, precision:-1, scale:-1}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:65536, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:-1, precision:25, scale:6}, {length:-1, precision:25, scale:6}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:-1, precision:20, scale:0}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}, {length:4000, precision:-1, scale:-1}, {length:128, precision:-1, scale:-1}], old_row_projector:[cnt:0], new_row_projector:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], is_total_quantity_log:1, is_ignore:0, is_batch_stmt:0, is_insert_up:0, is_table_api:0, tz_info:{id:-1, offset:28800, error_on_overlap_time:false}, table_param:{tenant_schema_version:1741597433156656, data_table:{table_id:519, schema_version:1741595950668896, table_type:0, index_type:0, index_status:0, shadow_rowkey_column_num:0, doc_id_col_id:18446744073709551615, fulltext_col_id:18446744073709551615, index_name:"", fts_parser_name:"", pk_name:"", columns:[cnt:29, {column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}], read_info:{schema_column_count:29, schema_rowkey_cnt:2, rowkey_cnt:4, trans_col_index:-1, mview_old_new_col_index:-1, group_idx_col_index:-1, seq_read_column_count:2, max_col_index:30, is_oracle_mode:false, cols_index:{rowkey_mode:false, for_memtable:false, array:{ObIArray:[cnt:29, 0, 1, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30], data:0x7f77b41ff850, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, memtable_cols_index:{rowkey_mode:false, for_memtable:true, array:{ObIArray:[cnt:29, 0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28], data:0x7f77b41ff8d0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}}, cols_desc:{ObIArray:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], data:0x7f77b41ff6f0, count:29, allocator:0x7f79d89f86b0, capacity:29, init_cnt:29}, cg_idxs:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, cols_extend:{ObIArray:[cnt:0], data:null, count:0, allocator:null, capacity:0, init_cnt:0}, has_all_column_group:true, datum_utils:{is_oracle_mode:false, rowkey_cnt:4, is_inited:true, is_oracle_mode:false}, cols_param:[{column_id:18, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:19, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:16, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:17, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"EXT":"CURRENT_TIMESTAMP"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:20, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:21, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:22, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:23, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:24, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:25, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:26, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:65536, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:27, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:28, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"BIGINT":0}, cur_default_value:{"BIGINT":0}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:29, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:30, meta_type:{type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:25, scale:6}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:31, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:32, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:33, meta_type:{type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:34, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:35, meta_type:{type:"BIGINT", collation:"binary", coercibility:"NUMERIC"}, order:0, accuracy:{length:-1, precision:20, scale:0}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:36, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:37, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:38, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:39, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:40, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:41, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:42, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:43, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:4000, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:false, is_nullable_for_read:false, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}, {column_id:44, meta_type:{type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"}, order:0, accuracy:{length:128, precision:-1, scale:-1}, orig_default_value:{"NULL":"NULL"}, cur_default_value:{"NULL":"NULL"}, is_nullable_for_write:true, is_nullable_for_read:true, is_gen_col:false, is_virtual_gen_col:false, is_gen_col_udf_expr:false, is_hidden:false, lob_chunk_size:262144}]}, lob_inrow_threshold:4096}, col_descs:[cnt:29, column_id=18 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=19 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=16 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=17 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=20 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=21 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=22 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=23 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=24 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=25 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=26 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=27 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=28 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=29 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=30 {type:"TIMESTAMP", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=31 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=32 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=33 {type:"BIGINT UNSIGNED", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=34 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=35 {type:"BIGINT", collation:"binary", coercibility:"NUMERIC"} order=0, column_id=36 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=37 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=38 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=39 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=40 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=41 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=42 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=43 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0, column_id=44 {type:"VARCHAR", collation:"utf8mb4_general_ci", coercibility:"INVALID"} order=0], col_map:{is_inited_:true, use_array_:true, shadow_use_array_:false, has_:true, has_shadow_:false}}, encrypt_meta:[cnt:0]}, table_rowkey_cids:[cnt:0], table_rowkey_types:[cnt:0]}, ins_rtdef:{ObDASBaseRtDef:{ObDASBaseRtDef:{op_type:2, children_cnt:0}, timeout_ts:1742176654506813, sql_mode:4194304, prelock:false, tenant_schema_version:-1, is_for_foreign_key_check:false, affected_rows:0}, need_fetch_conflict:false, is_duplicated:false, direct_insert_task_id:0, use_put:false, ddl_task_id:0}, insert_buffer:{mem_attr:tenant_id=1003, label=DASInsertBuffer, ctx_id=0, prio=0, buffer_memory:547, buffer_size:1, datum_store:NULL}}], remote_info:NULL}) [2025-03-17 09:57:04.609482] WDIAG [SQL.DAS] execute_das_task (ob_data_access_service.cpp:115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=821][errcode=-4016] failed to execute dist das task(ret=-4016) [2025-03-17 09:57:04.609491] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:391) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] failed to execute aggregated das task(ret=-4016, agg_task={server_:"192.168.1.62:2882", start_status_:0, tasks_.get_size():0, failed_tasks_.get_size():1, success_tasks_.get_size():0, save_ret_:0}, async=true) [2025-03-17 09:57:04.609502] WDIAG [SQL.DAS] execute_all_task (ob_das_ref.cpp:326) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=10][errcode=-4016] fail to execute all agg_tasks(ret=-4016) [2025-03-17 09:57:04.609508] WDIAG [SQL.ENG] submit_all_dml_task (ob_table_modify_op.cpp:1180) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] execute all dml das task failed(ret=-4016) [2025-03-17 09:57:04.609517] WDIAG [SQL.ENG] inner_get_next_row (ob_table_modify_op.cpp:1269) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] failed to submit the remaining dml tasks(ret=-4016) [2025-03-17 09:57:04.609526] WDIAG [SQL.ENG] get_next_row (ob_operator.cpp:1272) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] inner get next row failed(ret=-4016, type=24, op="PHY_INSERT", op_id=0) [2025-03-17 09:57:04.609537] WDIAG [SQL.EXE] get_next_row (ob_execute_result.cpp:148) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=10][errcode=-4016] get next row from operator failed(ret=-4016) [2025-03-17 09:57:04.609545] WDIAG [SQL] inner_get_next_row (ob_result_set.cpp:488) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] get next row from exec result failed(ret=-4016) [2025-03-17 09:57:04.609553] WDIAG [SQL] drive_dml_query (ob_result_set.cpp:1590) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=7][errcode=-4016] do dml query failed(ret=-4016) [2025-03-17 09:57:04.609561] WDIAG [SQL] open_result (ob_result_set.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] fail to drive dml query(ret=-4016) [2025-03-17 09:57:04.609569] WDIAG [SQL] open (ob_result_set.cpp:166) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:04.609575] WDIAG [SERVER] open (ob_inner_sql_result.cpp:155) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] open result set failed(ret=-4016) [2025-03-17 09:57:04.609584] WDIAG [SERVER] do_query (ob_inner_sql_connection.cpp:799) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] result set open failed(ret=-4016, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}) [2025-03-17 09:57:04.609593] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:944) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] execute failed(ret=-4016, tenant_id=1003, executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, retry_cnt=0, local_sys_schema_version=1741661297536408, local_tenant_schema_version=1741597433156656) [2025-03-17 09:57:04.609604] WDIAG [SERVER] after_func (ob_query_retry_ctrl.cpp:1066) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=11][errcode=-4016] [RETRY] check if need retry(v={force_local_retry:true, stmt_retry_times:0, local_retry_times:0, err_:-4016, err_:"OB_ERR_UNEXPECTED", retry_type:0, client_ret:-4016}, need_retry=false) [2025-03-17 09:57:04.609627] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=16] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f7974079118, id:3222524219, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2a26e0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:04.609660] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=20] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:4290}, to_seq_no={branch:0, seq:4}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634438} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:04.609686] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=22] end stmt(ret=0, tx_id=9634438, plain_select=false, stmt_type=2, savepoint={branch:0, seq:4}, tx_desc={this:0x7f7acf2a26e0, tx_id:{txid:9634438}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222524219, assoc_session_id:3222524219, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176624605533, active_ts:1742176624605533, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654605527, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468721106949, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624603417}], exec_info_reap_ts:1742176624603416, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624603413, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f7974079118, id:3222524219, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf2a26e0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.609732] WDIAG [SERVER] inner_close (ob_inner_sql_result.cpp:220) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=45][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:04.609738] WDIAG [SERVER] force_close (ob_inner_sql_result.cpp:200) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=6][errcode=-4016] result set close failed(ret=-4016) [2025-03-17 09:57:04.609743] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:949) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] failed to close result(close_ret=-4016, ret=-4016) [2025-03-17 09:57:04.609753] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:979) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=5][errcode=-4016] failed to process record(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, record_ret=-4016, ret=-4016) [2025-03-17 09:57:04.609760] WDIAG [SERVER] query (ob_inner_sql_connection.cpp:1016) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=6][errcode=-4016] failed to process final(executor={ObIExecutor:, sql:"INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')"}, aret=-4016, ret=-4016) [2025-03-17 09:57:04.609771] WDIAG [SERVER] execute_write_inner (ob_inner_sql_connection.cpp:1588) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=9][errcode=-4016] execute sql failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER')) [2025-03-17 09:57:04.609788] WDIAG [SERVER] retry_while_no_tenant_resource (ob_inner_sql_connection.cpp:1077) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=9][errcode=-4016] retry_while_no_tenant_resource failed(ret=-4016, tenant_id=1003) [2025-03-17 09:57:04.609797] WDIAG [SERVER] execute_write (ob_inner_sql_connection.cpp:1541) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] execute_write failed(ret=-4016, tenant_id=1003, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), is_user_sql=false) [2025-03-17 09:57:04.609807] WDIAG [COMMON.MYSQLP] write (ob_single_connection_proxy.cpp:136) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=9][errcode=-4016] execute sql failed(ret=-4016, sql=INSERT INTO __all_scheduler_job_run_detail_v2 (gmt_create, gmt_modified, job, time, code, message, job_class, database_name, owner, operation, status, req_start_date, job_name) VALUES (usec_to_time(1742176624605797), usec_to_time(1742176624605797), 10, usec_to_time(1742176624605797), -4012, X'636865636B206A6F622074696D656F7574', 'DEFAULT_JOB_CLASS', X'6F6365616E62617365', 'root@%', X'44424D535F53504D2E48414E444C455F53504D5F53544154535F4A4F425F50524F432829', 'SCHEDULED', usec_to_time(1741626000000000), 'SPM_STATS_MANAGER'), conn=0x7f7aeb4aa3a0) [2025-03-17 09:57:04.609816] WDIAG [SERVER] update_for_end (ob_dbms_sched_table_operator.cpp:387) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] fail to exec trans.write(tenant_id, sql2.ptr(), affected_rows)(ret=-4016) [2025-03-17 09:57:04.609883] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634438}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176654605527, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624606607}{self_ls_id:{id:1}, session_id:3222524219, associated_session_id:3222524219, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634438}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468721106949, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624606607, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634438} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.609956] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=61] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634438}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176654605527, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624606607}{self_ls_id:{id:1}, session_id:3222524219, associated_session_id:3222524219, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634438}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468721106949, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624606607, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634438} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=156 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.610009] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=48] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634438}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176624609878, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:04.610027] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=16] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf2a26e0, tx_id:{txid:9634438}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222524219, assoc_session_id:3222524219, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176624605533, active_ts:1742176624605533, commit_ts:-1, finish_ts:1742176624609879, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654605527, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468721106949, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:4}, last_touch_ts:1742176624603417}], exec_info_reap_ts:1742176624603416, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624603413, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:04.610106] WDIAG [SERVER] update_for_timeout (ob_dbms_sched_table_operator.cpp:336) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=31][errcode=-4016] fail to exec update_for_end(job_info, -4012, "check job timeout")(ret=-4016) [2025-03-17 09:57:04.610114] WDIAG [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:260) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=-4016] update for end failed for timeout job(ret=-4016) [2025-03-17 09:57:04.610123] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63414-0-0] [lt=8][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:04.629608] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117410][TimerWK3][T0][Y0-0000000000000000-0-0] [lt=28] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40416267468, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:04.631041] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:186) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=6] sql audit evict mem start(request_manager_->get_tenant_id()=1003, evict_low_mem_level=0, evict_high_mem_level=25769803, size_used=0, mem_used=29116416) [2025-03-17 09:57:04.631104] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539395][T1003_TimerWK3][T1003][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=1, elapse_time=63, size_used=0, mem_used=0) [2025-03-17 09:57:04.631140] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=9] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:04.635409] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117605][BatchIO][T0][Y0-0000000000000000-0-0] [lt=13] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.635970] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117609][BatchIO][T0][Y0-0000000000000000-0-0] [lt=24] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.635970] INFO [RPC.FRAME] rpc_easy_timer_cb (ob_net_easy.cpp:604) [117615][RpcUnix][T0][Y0-0000000000000000-0-0] [lt=16] [RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636019] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117610][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636166] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117611][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636224] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117608][BatchIO][T0][Y0-0000000000000000-0-0] [lt=17] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636387] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117604][BatchIO][T0][Y0-0000000000000000-0-0] [lt=25] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636443] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117606][BatchIO][T0][Y0-0000000000000000-0-0] [lt=19] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636569] INFO [RPC.FRAME] mysql_easy_timer_cb (ob_net_easy.cpp:677) [117613][MysqlUnix][T0][Y0-0000000000000000-0-0] [lt=24] [MYSQL EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.636569] INFO [RPC.FRAME] batch_rpc_easy_timer_cb (ob_net_easy.cpp:642) [117607][BatchIO][T0][Y0-0000000000000000-0-0] [lt=20] [BATCH_RPC EASY STAT](log_str=conn count=0/0, request done=0/0, request doing=0/0) [2025-03-17 09:57:04.641867] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [3539428][T1003_LSArchive][T1003][YB420A0B093E-00062FB7F8A54125-0-0] [lt=26] gc stale ls task succ [2025-03-17 09:57:04.662856] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=36] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:04.662889] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:04.662914] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=10] [PALF STAT GROUP LOG INFO](palf_id=1001, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=10, avg_log_batch_cnt=1, total_group_log_size=1220, avg_group_log_size=122) [2025-03-17 09:57:04.662943] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=28] [PALF STAT APPEND DATA SIZE](this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, append size=1220) [2025-03-17 09:57:04.663607] INFO [STORAGE.TRANS] alloc (ob_ts_response_handler.cpp:108) [117595][TsMgr][T1][Y0-0000000000000000-0-0] [lt=22] ts response task statistics(alloc_count=16165464, free_count=8082732) [2025-03-17 09:57:04.664021] INFO [STORAGE.TRANS] statistics (ob_gts_source.cpp:70) [117595][TsMgr][T0][Y0-0000000000000000-0-0] [lt=30] gts statistics(tenant_id=1004, gts_rpc_cnt=10, get_gts_cache_cnt=1082, get_gts_with_stc_cnt=3, try_get_gts_cache_cnt=0, try_get_gts_with_stc_cnt=0, wait_gts_elapse_cnt=3, try_wait_gts_elapse_cnt=0) [2025-03-17 09:57:04.664766] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=23] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=1342) [2025-03-17 09:57:04.664835] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=1342) [2025-03-17 09:57:04.664901] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=19] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=167850, hit_cnt=12, read_cnt=12, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:04.666539] WDIAG [OCCAM] runTimerTask (ob_occam_timer.h:163) [3539439][T1003_MultiVers][T1003][Y0-0000000000000000-0-0] [lt=63][errcode=0] task scheduled too delay(scheduled_delay=1475637, expected_run_ts_us_=1742176623190900, ObClockGenerator::getRealClock()=1742176624666538, ret=0, ret="OB_SUCCESS", *this={this:0x7f79da7fcf30, caller:ob_multi_version_garbage_collector.cpp:start:116, function_type:ZN9oceanbase19concurrency_control30ObMultiVersionGarbageCollector5startEvE6$_1073, timer_running_flag:1, total_running_count:1, func_shared_ptr_.ptr:0x7f79da7fd210, time_wheel:0x7f7aedd77990, thread_pool:0x7f7aedd77170, is_running:1, time_interval:60.00s, expected_run_time:09:57:03.190, task_priority:1, with_handle_protected:1}) [2025-03-17 09:57:04.666639] INFO [MVCC] repeat_study (ob_multi_version_garbage_collector.cpp:197) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=11] skip repeat study(ret=0, this={this:0x7f79a51df030, last_study_timestamp:1742176264695128, last_refresh_timestamp:1742176564663414, last_reclaim_timestamp:1742176264697208, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176264250519000, v:0}, is_inited:true}, current_timestamp=1742176624666637, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:57:04.667525] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=35] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1742176264640354000, snapshot_type=1, create_time=1742176264641075, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.667553] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=26] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1742176264250519000, snapshot_type=2, create_time=1742176264641075, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.667560] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=6] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=1742176264582837000, snapshot_type=3, create_time=1742176264641075, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.667565] INFO [MVCC] collect (ob_multi_version_garbage_collector.cpp:709) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=4] multi version garbage colloector collects successfully(sql= select snapshot_version, snapshot_type, svr_ip, svr_port, create_time, status from __all_reserved_snapshot where tenant_id = 1003 , snapshot_version=4611686018427387903, snapshot_type=4, create_time=1742176264641075, addr="192.168.1.62:2882", gc_status=0) [2025-03-17 09:57:04.667677] INFO [MVCC] decide_reserved_snapshot_version_ (ob_multi_version_garbage_collector.cpp:620) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=5] succeed to update global reserved snapshot(ret=0, this={this:0x7f79a51df030, last_study_timestamp:1742176264695128, last_refresh_timestamp:1742176564663414, last_reclaim_timestamp:1742176264697208, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176264250519000, v:0}, is_inited:true}, global_reserved_snapshot_={val:1742176264250519000, v:0}, reserved_snapshot={val:1742176264250519000, v:0}) [2025-03-17 09:57:04.667693] INFO [MVCC] refresh_ (ob_multi_version_garbage_collector.cpp:554) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=15] multi-version garbage collector refresh successfully(this={this:0x7f79a51df030, last_study_timestamp:1742176264695128, last_refresh_timestamp:1742176564663414, last_reclaim_timestamp:1742176264697208, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176264250519000, v:0}, is_inited:true}, collector={reserved_snapshot_version:{val:1742176264250519000, v:0}, reserved_snapshot_type:2, reserved_status:-1, reserved_create_time:1742176264641075, reserved_addr:"192.168.1.62:2882", is_this_server_disabled:false, status:0}) [2025-03-17 09:57:04.667705] INFO [MVCC] repeat_refresh (ob_multi_version_garbage_collector.cpp:232) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=11] repeat refresh successfully(ret=0, this={this:0x7f79a51df030, last_study_timestamp:1742176264695128, last_refresh_timestamp:1742176624667704, last_reclaim_timestamp:1742176264697208, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176264250519000, v:0}, is_inited:true}, current_timestamp=1742176624666675, GARBAGE_COLLECT_RETRY_INTERVAL=60000000) [2025-03-17 09:57:04.667715] INFO [MVCC] repeat_reclaim (ob_multi_version_garbage_collector.cpp:267) [3539438][T1003_Occam][T1003][Y0-0000000000000000-0-0] [lt=9] skip repeat reclaim(ret=0, this={this:0x7f79a51df030, last_study_timestamp:1742176264695128, last_refresh_timestamp:1742176624667704, last_reclaim_timestamp:1742176264697208, last_sstable_overflow_timestamp:0, has_error_when_study:false, refresh_error_too_long:false, has_error_when_reclaim:false, gc_is_disabled:false, global_reserved_snapshot:{val:1742176264250519000, v:0}, is_inited:true}, current_timestamp=1742176624667715, GARBAGE_COLLECT_EXEC_INTERVAL=600000000) [2025-03-17 09:57:04.671193] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=34, len=263, pos=233) [2025-03-17 09:57:04.671701] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=24][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=341, pos=295) [2025-03-17 09:57:04.672370] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=14][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=53, len=281, pos=252) [2025-03-17 09:57:04.672990] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=15][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=412, pos=297) [2025-03-17 09:57:04.673620] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=46, len=268, pos=238) [2025-03-17 09:57:04.674127] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=10][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=337, pos=286) [2025-03-17 09:57:04.674350] INFO [ARCHIVE] gc_stale_ls_task_ (ob_ls_mgr.cpp:586) [117822][T1_LSArchiveMgr][T1][YB420A0B093E-00062FB7EE654125-0-0] [lt=21] gc stale ls task succ [2025-03-17 09:57:04.674908] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=59, len=295, pos=265) [2025-03-17 09:57:04.675690] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5728-0-0] [lt=20][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=12, len=165, pos=142) [2025-03-17 09:57:04.677282] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=8) [2025-03-17 09:57:04.682186] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=20][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162423032}], scn_array=[cnt:1, {val:1742176624163293004, v:0}], curr_size=122, accum_size=366, time_cost=19242) [2025-03-17 09:57:04.683997] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.693135] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=23][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=81, len=349, pos=321) [2025-03-17 09:57:04.693681] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=50][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=74, len=327, pos=297) [2025-03-17 09:57:04.694112] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=13][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=2, len=65, pos=48) [2025-03-17 09:57:04.694657] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=15][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=14, len=176, pos=152) [2025-03-17 09:57:04.695086] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=9][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=336, pos=298) [2025-03-17 09:57:04.695572] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=6][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=75, len=740, pos=287) [2025-03-17 09:57:04.696090] WDIAG [RPC.OBMYSQL] serialize (ob_mysql_row.cpp:56) [118874][T1_L0_G10000][T1][YB420A0B093E-00062FB7F06F5729-0-0] [lt=8][errcode=-4019] failed to encode cell(ret=-4019, cell_idx=1, len=22, pos=15) [2025-03-17 09:57:04.696532] INFO [SHARE] run_loop_ (ob_bg_thread_monitor.cpp:331) [117929][BGThreadMonitor][T0][Y0-0000000000000000-0-0] [lt=28] current monitor number(seq_=-1) [2025-03-17 09:57:04.704105] INFO [PALF] inner_append_log (palf_handle_impl.cpp:2231) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=34] [PALF STAT INNER APPEND LOG SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, accum_size=8346) [2025-03-17 09:57:04.704171] INFO [LIB] stat (utility.h:1227) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=59] [PALF STAT IO TASK IN QUEUE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=20, this=0x7f79a49ea530, extra_info="") [2025-03-17 09:57:04.704201] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=24] [PALF STAT FS CB EXCUTE COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=9, this=0x7f79aaff16a0, extra_info="palf_id:1") [2025-03-17 09:57:04.704230] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=29] [PALF STAT LOG LIFE TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=21543, this=0x7f79aaff18d0, extra_info="palf_id:1") [2025-03-17 09:57:04.704239] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=7] [PALF STAT LOG TASK TIME](palf_id=1, self="192.168.1.62:2882", total_slide_log_cnt=13, avg_log_gen_to_freeze_time=900, avg_log_gen_to_submit_time=912, avg_log_submit_to_flush_time=20610, avg_log_submit_to_slide_time=20631) [2025-03-17 09:57:04.704260] INFO [LIB] stat (utility.h:1210) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=10] [PALF STAT FLUSH CB COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=43, this=0x7f79aaffbe60, extra_info="palf_id:1") [2025-03-17 09:57:04.704277] INFO [PALF] read (log_cache.cpp:103) [3539404][T1003_LogShared][T1003][Y0-0000000000000000-0-0] [lt=31] [PALF STAT HOT CACHE HIT RATE](palf_id=1, read_size=8346, hit_cnt=13, read_cnt=13, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:04.716862] INFO [SERVER] scheduler_job (ob_dbms_sched_job_master.cpp:257) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=8] job is running now, retry later(job_info={tenant_id_:1003, user_id_:18446744073709551615, database_id_:18446744073709551615, job_:9, job_name_:"ASYNC_GATHER_STATS_JOB_PROC", job_style_:"REGULAR", job_type_:"", lowner_:"root@%", powner_:"root@%", cowner_:"oceanbase", last_modify_:1742045952860334, start_date_:1741596852858963, last_date_:1742045052860340, this_date_:1742045952860334, next_date_:1742046852858963, end_date_:64060560000000000, total_:92572320, interval_:"FREQ=MINUTELY; INTERVAL=15", repeat_interval_:"FREQ=MINUTELY; INTERVAL=15", failures_:0, flag_:0, what_:"DBMS_STATS.ASYNC_GATHER_STATS_JOB_PROC(600000000)", nlsenv_:"", charenv_:"", field1_:"", scheduler_flags_:0, enabled_:true, auto_drop_:false, max_run_duration_:600, interval_ts_:900000000, max_failures_:0, state_:"SCHEDULED"}) [2025-03-17 09:57:04.718527] WDIAG [STORAGE] deserialize (ob_micro_block_header.cpp:301) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=18][errcode=-4016] invalid header to deserialize(ret=-4016, buf_len=1611, pos=0, header={magic:0, version:0, header_size:0, header_checksum:0, column_count:0, rowkey_column_count:0, has_column_checksum:0, row_count:0, row_store_type:0, opt:0, var_column_count:0, compressor_type:0, row_offset:0, original_length:0, max_merged_trans_version:0, min_merged_trans_version:0, data_length:0, data_zlength:0, data_checksum:0, column_checksums:null, single_version_rows:0, contain_uncommitted_rows:0, is_last_row_last_flag:0, is_valid():false}) [2025-03-17 09:57:04.718559] EDIAG [STORAGE] process_block (ob_micro_block_cache.cpp:461) [3539397][T1003_DiskCB0][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=30][errcode=-4016] Fail to deserialize record header(ret=-4016, block_id={[ver=1,mode=0,seq=35119][2nd=35120]}, offset=1581402) BACKTRACE:0x810107f 0x7b89ba6 0x7b891d3 0x7b88c7c 0x7b8891c 0x7b8870a 0x1795ed0d 0x7af2023 0x7af0cb4 0x7aefc53 0x1f9588e5 0x1f95655e 0x7f7b46c081ca 0x7f7b468398d3 [2025-03-17 09:57:04.718584] WDIAG [COMMON] wait (ob_io_define.cpp:1885) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=41][errcode=-4016] REACH SYSLOG RATE LIMIT [bandwidth] [2025-03-17 09:57:04.718982] INFO [DETECT] maintain_deadlock_info_when_end_stmt (ob_trans_deadlock_adapter.cpp:662) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=0] maintain deadlock info(step=2, ret=0, ret="OB_SUCCESS", exec_ctx.get_errcode()=-4016, exec_ctx.get_errcode()="OB_ERR_UNEXPECTED", session={this:0x7f79740701f8, id:3222517689, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3bfbc0}, desc=NULL, is_rollback=true, conflict_txs=[]) [2025-03-17 09:57:04.719032] INFO [STORAGE.TRANS] rollback (ob_memtable_context.cpp:853) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=35] memtable handle rollback to successfuly(from_seq_no={branch:0, seq:1111}, to_seq_no={branch:0, seq:1109}, remove_cnt=1, elapsed=0, this={ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634440} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}}) [2025-03-17 09:57:04.719077] INFO [SQL.EXE] end_stmt (ob_sql_trans_control.cpp:1503) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=40] end stmt(ret=0, tx_id=9634440, plain_select=false, stmt_type=2, savepoint={branch:0, seq:1109}, tx_desc={this:0x7f7acf3bfbc0, tx_id:{txid:9634440}, state:2, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517689, assoc_session_id:3222517689, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:6, alloc_ts:1742176624716847, active_ts:1742176624716847, commit_ts:-1, finish_ts:-1, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654716841, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468832276496, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:1109}, last_touch_ts:1742176624716848}], exec_info_reap_ts:1742176624716847, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624715739, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:0, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}, trans_result={incomplete:false, parts:[], touched_ls_list:[], conflict_txs:[]}, rollback=true, need_rollback=true, session={this:0x7f79740701f8, id:3222517689, deser:false, tenant:"sys", tenant_id:1, effective_tenant:"sys", effective_tenant_id:1003, database:"oceanbase", user:"root@%", consistency_level:3, session_state:0, autocommit:true, tx:0x7f7acf3bfbc0}, exec_ctx.get_errcode()=-4016) [2025-03-17 09:57:04.719209] WDIAG [STORAGE.TRANS] do_local_abort_tx_ (ob_trans_part_ctx.cpp:9171) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=0][errcode=0] do_local_abort_tx_(ret=0, ret="OB_SUCCESS", *this={this:0x7f79a5f09950, ref:2, trans_id:{txid:9634440}, tenant_id:1003, is_exiting:false, trans_expired_time:1742176654716841, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624716847}{self_ls_id:{id:1}, session_id:3222517689, associated_session_id:3222517689, part_trans_action:2, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634440}, ref_cnt:1, state:"RUNNING", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468832276496, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624717912, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:0, v:0} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=0 tx_status=0 is_readonly=false ref=0 trans_id={txid:9634440} ls_id=1 row_callback[alloc:1, free:0, unsubmit:1] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=2 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=0, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(1,2,0,0,3,1,1,0)|]}} -{ -{downstream_state:"INIT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.719271] INFO [STORAGE.TRANS] abort_ (ob_trans_part_ctx.cpp:2115) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=49] tx abort(ret=0, reason=-6002, reason_str="OB_TRANS_ROLLBACKED", this={this:0x7f79a5f09950, ref:1, trans_id:{txid:9634440}, tenant_id:1003, is_exiting:true, trans_expired_time:1742176654716841, cluster_version:17180067072, trans_need_wait_wrap:{receive_gts_ts_:[mts=0], need_wait_interval_us:0}, stc:[mts=0], ctx_create_time:1742176624716847}{self_ls_id:{id:1}, session_id:3222517689, associated_session_id:3222517689, part_trans_action:4, pending_write:0, 2pc_role:"UNKNOWN", ctx_tx_data_:{ctx_mgr_:0x7f79c0804030, tx_data_guard_:{tx_data:{tx_id:{txid:9634440}, ref_cnt:1, state:"ABORT", commit_version:{val:18446744073709551615, v:3}, start_scn:{val:18446744073709551615, v:3}, end_scn:{val:18446744073709551615, v:3}, op_guard:{tx_data_op_:null}}}, read_only_:false}, role_state_:0, create_ctx_scn_:{val:18446744073709551615, v:3}, ctx_source:1, epoch_:409468832276496, replay_completeness_:1, upstream_state:"INIT", collected:[], rec_log_ts:{val:18446744073709551615, v:3}, prev_rec_log_ts:{val:18446744073709551615, v:3}, lastest_snapshot:{val:18446744073709551615, v:3}, last_request_ts:1742176624717912, block_frozen_memtable:null, max_2pc_commit_scn:{val:18446744073709551615, v:3}, mt_ctx_:{ObIMvccCtx={alloc_type=0 ctx_descriptor=0 min_table_version=1741595949571232 max_table_version=1741595950668896 trans_version={val:4611686018427387903, v:0} commit_version={val:18446744073709551615, v:3} lock_wait_start_ts=0 replay_compact_version={val:0, v:0}} end_code=-6002 tx_status=1 is_readonly=false ref=0 trans_id={txid:9634440} ls_id=1 row_callback[alloc:1, free:1, unsubmit:0] redo[fill:0,sync_succ:0, sync_fail:0] main_list_len=0 pending_log_size=166 callback_list:{cnt=1 stat:[tx_end=2, rollback_to=1, fast_commit=0, remove_memtable=0, ext_info_log=0] detail:[(log_epoch,length,logged,synced,appended,removed,unlog_removed,branch_removed)|0:(-1,0,0,0,3,3,3,0)|]}} -{ -{force_abort:1 }-, -{downstream_state:"ABORT", upstream:{id:-1}, participants:[], redo_log_no:0, scheduler:"192.168.1.62:2882", prepare_version:{val:18446744073709551615, v:3}, trans_type:0, next_log_entry_no:0, max_applied_log_ts:{val:18446744073709551615, v:3}, max_appling_log_ts:{val:18446744073709551615, v:3}, max_applying_part_log_no:9223372036854775807, max_submitted_seq_no:0, checksum:[0], checksum_scn:[{val:0, v:0}], need_checksum:true, data_complete:false, is_dup_tx:false, exec_epoch:0, }- }-}) [2025-03-17 09:57:04.719333] INFO [STORAGE.TRANS] handle_trans_abort_request (ob_trans_service_v4.cpp:2093) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=56] handle trans abort request(ret=0, abort_req={txMsg:{type:22, cluster_version:17180067072, tenant_id:1003, tx_id:{txid:9634440}, receiver:{id:1}, sender:{id:9223372036854775807}, sender_addr:"192.168.1.62:2882", epoch:-1, transfer_epoch:-1, request_id:7, timestamp:1742176624719205, cluster_id:1741272511}, reason:-6002}) [2025-03-17 09:57:04.719349] INFO [STORAGE.TRANS] rollback_tx (ob_tx_api.cpp:374) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=14] rollback tx(ret=0, *this={is_inited_:true, tenant_id_:1003, this:0x7f79a5404030}, tx={this:0x7f7acf3bfbc0, tx_id:{txid:9634440}, state:7, addr:"192.168.1.62:2882", tenant_id:1003, session_id:3222517689, assoc_session_id:3222517689, xid:NULL, xa_mode:"", xa_start_addr:"0.0.0.0:0", access_mode:0, tx_consistency_type:0, isolation:1, snapshot_version:{val:18446744073709551615, v:3}, snapshot_scn:0, active_scn:{branch:0, seq:1}, op_sn:7, alloc_ts:1742176624716847, active_ts:1742176624716847, commit_ts:-1, finish_ts:1742176624718975, timeout_us:29999994, lock_timeout_us:-1, expire_ts:1742176654716841, coord_id:{id:-1}, parts:[{id:{id:1}, addr:"192.168.1.62:2882", epoch:409468832276496, first_scn:{branch:0, seq:3}, last_scn:{branch:0, seq:1109}, last_touch_ts:1742176624716848}], exec_info_reap_ts:1742176624716847, commit_version:{val:18446744073709551615, v:3}, commit_times:0, commit_cb:null, cluster_id:1741272511, cluster_version:17180067072, seq_base:1742176624715739, flags_.SHADOW:false, flags_.INTERRUPTED:false, flags_.BLOCK:false, flags_.REPLICA:false, can_elr:true, conflict_txs:[], abort_cause:-6002, commit_expire_ts:0, commit_task_.is_registered():false, modified_tables:[], ref:2}) [2025-03-17 09:57:04.719417] WDIAG [SERVER] scheduler (ob_dbms_sched_job_master.cpp:217) [3539440][T1003_DBMSSched][T1003][YB420A0B093E-00062FB95BF63415-0-0] [lt=0][errcode=0] fail to scheduler single dbms sched job(ret=0, tmp_ret=-4016) [2025-03-17 09:57:04.725353] INFO [SQL] alloc_value (ob_sql_session_mgr.cpp:272) [118873][T1_L0_G10000][T1][YB420A0B093E-00062FB8DBD9BBCC-0-0] [lt=8] alloc_session_count(alloc_total_count=122130000) [2025-03-17 09:57:04.737441] INFO [PALF] try_recycle_blocks (palf_env_impl.cpp:817) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=12] LOG_DISK_OPTION(disk_options_wrapper_={disk_opts_for_stopping_writing:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, disk_opts_for_recycling_blocks:{log_disk_size(MB):4915, log_disk_utilization_threshold(%):80, log_disk_utilization_limit_threshold(%):95, log_disk_throttling_percentage(%):60, log_disk_throttling_maximum_duration(s):7200, log_writer_parallelism:3}, status:1, cur_unrecyclable_log_disk_size(MB):218, sequence:1}) [2025-03-17 09:57:04.751660] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=25][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859737778}], scn_array=[cnt:1, {val:1742176624735023000, v:0}], curr_size=588, accum_size=588, time_cost=15632) [2025-03-17 09:57:04.763077] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [117791][T1_IOWorker][T1][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:5463555149}], scn_array=[cnt:1, {val:1742176624749930000, v:0}], curr_size=587, accum_size=2713, time_cost=12465) [2025-03-17 09:57:04.779171] INFO [STORAGE] scheduler_ls_ha_handler_ (ob_storage_ha_service.cpp:196) [117828][T1_HAService][T1][Y0-0000000000000000-0-0] [lt=34] start do ls ha handler(ls_id_array_=[{id:1}]) [2025-03-17 09:57:04.783645] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162423154}], scn_array=[cnt:1, {val:1742176624663416000, v:0}], curr_size=122, accum_size=488, time_cost=20671) [2025-03-17 09:57:04.784077] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=23][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.786787] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=29][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872709598}], scn_array=[cnt:1, {val:1742176624663416000, v:0}], curr_size=122, accum_size=122, time_cost=23831) [2025-03-17 09:57:04.808440] INFO [SQL] free_value (ob_sql_session_mgr.cpp:325) [118872][T1_L0_G10000][T1][YB420A0B093E-00062FB937B85904-0-0] [lt=6] free_session_count(free_total_count=122130000) [2025-03-17 09:57:04.829972] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=19] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40428850380, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:04.863176] INFO try_report (ob_tc_stat.cpp:35) [117636][][T0][Y0-0000000000000000-0-0] [lt=20] QSched: |cfg: root:1,x,0 net_in:1,1203M,0 net_out:1,1249M,0 disk:1,x,0 t500m0:1,x,0 t500m0gd:1,x,0 t500m1:1,x,0 t500m1gd:1,x,0 t500m2:1,x,0 t500m2gd:1,x,0 t1m0:4,x,0 t1m0gd:1,x,0 t1m1:4,x,0 t1m1gd:1,x,0 t1m2:4,x,x t1m2gd:1,x,0 t1m0g0:100,x,0 ips:x ibw:x t1m1g0:100,x,0 ops:x obw:x t1m2g0:100,x,x t1m0g10000:100,x,0 t1m1g10000:100,x,0 t1m2g10000:100,x,0 t1004m0:8,x,0 t1004m0gd:1,x,0 t1004m1:8,x,0 t1004m1gd:1,x,0 t1004m2:8,x,x t1004m2gd:1,x,0 t1004m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x ips:x ibw:x t1004m1g0:100,x,0 ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x ops:x obw:x t1004m2g0:100,x,x t1003m0:8,x,0 t1003m0gd:1,x,0 t1003m1:8,x,0 t1003m1gd:1,x,0 t1003m2:8,x,x t1003m2gd:1,x,0 t1003m0g0:100,x,0 ips:x ibw:x ips:x ibw:x ips:x ibw:x t1003m1g0:100,x,0 ops:x obw:x t1003m2g0:100,x,x|stat: root:426K/65:14760 net_out:162K/1:8521 disk:264K/64:14858 t1m2:80K/20:13946 t1m2g0:80K/20:13946 t1004m1:162K/1:8521 t1004m2:88K/22:16660 t1004m1g0:162K/1:8521 t1004m2g0:88K/22:16660 t1003m2:96K/22:13884 t1003m2g0:96K/22:13884 |q0: 0 0 0 0 |q1: 0 0 0 0 [2025-03-17 09:57:04.877415] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=25] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=7) [2025-03-17 09:57:04.881686] INFO [PALF] submit_log (palf_handle_impl.cpp:458) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=33] [PALF STAT APPEND DATA SIZE](this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, append size=3544) [2025-03-17 09:57:04.883743] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=39][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162423276}], scn_array=[cnt:1, {val:1742176624663416001, v:0}], curr_size=122, accum_size=610, time_cost=20757) [2025-03-17 09:57:04.883732] INFO [PALF] inner_write_once_ (log_block_handler.cpp:410) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=43] [PALF STAT WRITE LOG INFO TO DISK](ret=0, offset=351832, this={dio_aligned_buf:{buf_write_offset:3794, buf_padding_size:302, align_size:4096, aligned_buf_size:3682304, aligned_used_ts:3718210, truncate_used_ts:1456722}, log_block_size:67108864, io_fd:{first_id:-1, second_id:198, third_id:-1, fd_id:-1, slot_version:-1, device_handle:0x7f7b3333a080}}, aligned_buf_len=4096, aligned_block_offset=348160, buf_len=122, total_write_size=13957, total_write_size_after_dio=430080, ob_pwrite_used_ts=1684388, count=102, each_pwrite_cost=16513) [2025-03-17 09:57:04.883780] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539216][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=45][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:872709720}], scn_array=[cnt:1, {val:1742176624663416001, v:0}], curr_size=122, accum_size=244, time_cost=20822) [2025-03-17 09:57:04.884155] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=26][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.899479] INFO [SHARE] update_decay_factor_ (ob_throttle_unit.ipp:284) [3539427][T1003_TenantWea][T1003][YB420A0B093E-00062FB95B26709F-0-0] [lt=34] [Throttle] Update Throttle Unit Config(is_adaptive_update=true, N=1.205825047492980957e+02, this=0x7f79a767b208, enable_adaptive_limit_=true, Unit Name=TxShare, Config Specify Resource Limit(MB)=819, Resource Limit(MB)=602, Throttle Trigger(MB)=361, Throttle Percentage=60, Max Duration(us)=7200000000, Decay Factor=1.339928752955151765e+02) [2025-03-17 09:57:04.903891] INFO [STORAGE] diagnose_for_suggestion (ob_compaction_suggestion.cpp:458) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] [COMPACTION DAG STATUS] (start_time=1742176614902178, end_time=1742176624903889, dag_status={}) [2025-03-17 09:57:04.903924] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=31] dump_dag_status(priority="PRIO_COMPACTION_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903933] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=9] dump_dag_status(priority="PRIO_HA_HIGH", limits=8, running_task=0, adaptive_task_limit=8, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903939] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(priority="PRIO_COMPACTION_MID", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903943] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(priority="PRIO_HA_MID", limits=5, running_task=0, adaptive_task_limit=5, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903955] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=12] dump_dag_status(priority="PRIO_COMPACTION_LOW", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903962] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(priority="PRIO_HA_LOW", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903969] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903975] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_DDL_HIGH", limits=6, running_task=0, adaptive_task_limit=6, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903982] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:2861) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(priority="PRIO_TTL", limits=2, running_task=0, adaptive_task_limit=2, ready_dag_count=0, waiting_dag_count=0, rank_dag_count=0) [2025-03-17 09:57:04.903988] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:0, sys_task_type:3, dag_type_str:"MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.903998] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:2, sys_task_type:5, dag_type_str:"MINOR_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904004] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"MAJOR_MERGE/MEDIUM_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904008] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_BATCH_EXECUTE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904016] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_PREPARE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904023] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_SCHEDULE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904028] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"CO_MERGE_FINISH", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904033] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:0, sys_task_type:4, dag_type_str:"TX_TABLE_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904045] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=11] dump_dag_status(type={init_dag_prio:0, sys_task_type:19, dag_type_str:"MDS_MINI_MERGE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904050] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"VERIFY_CKM", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904055] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:4, sys_task_type:23, dag_type_str:"BATCH_FREEZE", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904059] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"UPDATE_SKIP_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904064] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:4, sys_task_type:6, dag_type_str:"REFRESH_MAJOR", dag_module_str:"COMPACTION"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904069] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_COMPLEMENT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904074] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"UNIQUE_CHECK", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904078] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"SQL_BUILD_INDEX", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904083] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:7, sys_task_type:12, dag_type_str:"DDL_KV_MERGE", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904088] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:2, dag_type_str:"DDL_DEL_LOB_META", dag_module_str:"DROP_VEC_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904093] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904098] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:6, sys_task_type:25, dag_type_str:"DDL_TABLET_SPLIT", dag_module_str:"DDL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904102] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904107] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATE_WARMUP", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904112] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"WAIT_DATA_READY", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904117] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_COMPLETE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904122] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904126] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904131] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"SYS_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904136] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904140] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"DATA_TABLETS_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904160] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=20] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_GROUP_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904165] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"MIGRATION_FINISH", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904170] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"INITIAL_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904175] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"START_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904180] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"FINISH_PREPARE_MIGRATION", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904185] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:1, dag_type_str:"TABLET_CHECKE_CONVERT", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904190] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:3, sys_task_type:1, dag_type_str:"FAST_MIGRATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904195] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:1, dag_type_str:"VALIDATE", dag_module_str:"MIGRATE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904199] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"TABLET_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904204] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:16, dag_type_str:"FINISH_BACKFILL_TX", dag_module_str:"BACKFILL_TX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904209] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_META", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904216] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_PREPARE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904226] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_FINISH", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904231] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_DATA", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904239] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"PREFETCH_BACKUP_INFO", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904246] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_INDEX_REBUILD", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904254] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904262] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS_GROUP", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904269] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_COMPLEMENT_LOG_LS", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904276] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"INITIAL_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904283] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"START_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904291] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"BACKUP_TABLET_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904298] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:8, dag_type_str:"FINISH_BACKUP_FUSE", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904305] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:10, dag_type_str:"BACKUP_BACKUPSET", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904312] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:5, sys_task_type:11, dag_type_str:"BACKUP_ARCHIVELOG", dag_module_str:"BACKUP"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904319] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904327] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904334] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"SYS_TABLETS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904341] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"DATA_TABLETS_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904348] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_GROUP_META_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904355] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_LS_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904360] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"INITIAL_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904365] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"START_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904376] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=10] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"FINISH_TABLET_GROUP_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904381] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:14, dag_type_str:"TABLET_RESTORE", dag_module_str:"RESTORE"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904385] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status(type={init_dag_prio:5, sys_task_type:15, dag_type_str:"BACKUP_CLEAN", dag_module_str:"BACKUP_CLEAN"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904390] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:3, sys_task_type:17, dag_type_str:"REMOVE_MEMBER", dag_module_str:"REMOVE_MEMBER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904395] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BACKFILL_TX", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904400] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_REPLACE_TABLE", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904408] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:1, sys_task_type:18, dag_type_str:"TRANSFER_BUILD_TABLET_INFO", dag_module_str:"TRANSFER"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904415] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:8, sys_task_type:20, dag_type_str:"TTL_DELTE_DAG", dag_module_str:"TTL"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904433] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=18] dump_dag_status(type={init_dag_prio:3, sys_task_type:21, dag_type_str:"TENANT_SNAPSHOT_CREATE", dag_module_str:"TSNAP_CR8"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904442] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status(type={init_dag_prio:5, sys_task_type:22, dag_type_str:"TENANT_SNAPSHOT_GC", dag_module_str:"TSNAP_GC"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904449] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4328) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status(type={init_dag_prio:8, sys_task_type:24, dag_type_str:"VECTOR_INDEX_DAG", dag_module_str:"VECTOR_INDEX"}, dag_count=0, running_dag_count=0, added_dag_count=0, scheduled_dag_count=0, scheduled_task_count=0, scheduled_data_size=0) [2025-03-17 09:57:04.904454] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3635) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](running_dag_net_map_size=0, blocking_dag_net_list_size=0, running_dag_net_list_size=0) [2025-03-17 09:57:04.904461] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_MIGRATION", dag_count=0) [2025-03-17 09:57:04.904469] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=7] dump_dag_status[DAG_NET](type="DAG_NET_PREPARE_MIGRATION", dag_count=0) [2025-03-17 09:57:04.904477] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=8] dump_dag_status[DAG_NET](type="DAG_NET_COMPLETE_MIGRATION", dag_count=0) [2025-03-17 09:57:04.904481] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER", dag_count=0) [2025-03-17 09:57:04.904486] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=5] dump_dag_status[DAG_NET](type="DAG_NET_BACKUP", dag_count=0) [2025-03-17 09:57:04.904490] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=4] dump_dag_status[DAG_NET](type="DAG_NET_RESTORE", dag_count=0) [2025-03-17 09:57:04.904496] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_BACKUP_CLEAN", dag_count=0) [2025-03-17 09:57:04.904499] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=3] dump_dag_status[DAG_NET](type="DAG_NET_TYPE_CO_MAJOR", dag_count=0) [2025-03-17 09:57:04.904505] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:3638) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status[DAG_NET](type="DAG_NET_TRANSFER_BACKFILL_TX", dag_count=0) [2025-03-17 09:57:04.904512] INFO [COMMON] dump_dag_status (ob_tenant_dag_scheduler.cpp:4342) [3539199][T1004_DagSchedu][T1004][Y0-0000000000000000-0-0] [lt=6] dump_dag_status(dag_cnt=0, total_worker_cnt_=0, total_running_task_cnt=0, work_thread_num_=43, scheduled_task_cnt=0) [2025-03-17 09:57:04.920986] INFO [SQL.QRR] runTimerTask (ob_udr_mgr.cpp:93) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=26] run rewrite rule refresh task(rule_mgr_->tenant_id_=1003) [2025-03-17 09:57:04.922027] INFO [STORAGE.TRANS] get_number (ob_id_service.cpp:393) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=28] get number(ret=0, service_type_=0, range=1, base_id=1742176624921855000, start_id=1742176624921855000, end_id=1742176624921855001) [2025-03-17 09:57:04.933215] INFO [LIB] stat (utility.h:1210) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=35] [PALF STAT WRITE LOG COST TIME](cur_stat_count=12, stat_interval=1000000, avg (count/cost)=21444, this=0x7f79aaffbc30, extra_info="palf_id:1") [2025-03-17 09:57:04.933247] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=30][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859738366}], scn_array=[cnt:1, {val:1742176624899421000, v:0}], curr_size=588, accum_size=1176, time_cost=33418) [2025-03-17 09:57:04.933465] INFO [STORAGE.TRANS] persist_version_if_need_ (ob_tenant_weak_read_cluster_service.cpp:338) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=44] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] persist CLUSTER version range succ(tenant_id=1003, new_min_version={val:1742176624698558000, v:0}, new_max_version={val:1742176624698558000, v:0}, last_min_version={val:1742176624498673000, v:0}, last_max_version={val:1742176624498673000, v:0}, total_time_us=35320, delta_us=234906) [2025-03-17 09:57:04.933488] INFO [STORAGE.TRANS] update_cluster_version (ob_tenant_weak_read_cluster_service.cpp:697) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=21] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] update version(tenant_id=1003, current_version={val:1742176624698558000, v:0}, min_version={val:1742176624698558000, v:0}, max_version={val:1742176624698558000, v:0}, in_service=true, leader_epoch=1) [2025-03-17 09:57:04.941143] WDIAG [SERVER] get_network_speed_from_sysfs (ob_server.cpp:3183) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=24][errcode=-4002] get invalid Ethernet speed, use default(devname="ens16") [2025-03-17 09:57:04.941183] WDIAG [SERVER] runTimerTask (ob_server.cpp:3716) [117408][TimerWK1][T0][Y0-0000000000000000-0-0] [lt=39][errcode=-4002] ObRefreshNetworkSpeedTask reload bandwidth throttle limit failed(ret=-4002, ret="OB_INVALID_ARGUMENT") [2025-03-17 09:57:04.941266] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:504) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=17] try inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312811559665, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:57:04.941333] INFO [LIB] do_thread_recycle (threads.cpp:163) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=37] recycle one thread(this=0x7f7b319af170, total=4, remain=3) [2025-03-17 09:57:04.941346] WDIAG [LIB] try_wait (thread.cpp:249) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=13][errcode=-4023] pthread_tryjoin_np failed(pret=16, errno=11, ret=-4023) [2025-03-17 09:57:04.941357] INFO [LIB] do_thread_recycle (threads.cpp:146) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=10] try_wait return eagain(this=0x7f7b319af170, thread=0x7f7b05619ed0) [2025-03-17 09:57:04.941366] INFO [COMMON] try_inc_thread_count (ob_dynamic_thread_pool.cpp:509) [117406][qth_mgr][T0][Y0-0000000000000000-0-0] [lt=8] inc thread count(*this={name:IO_SYNC_CH, this:0x7f7b319af170, min_thread_cnt:1, max_thread_cnt:128, running_thread_cnt:1, threads_idle_time:1312811559665, tenant_id:500}, cur_thread_count=4, cnt=-1, new_thread_count=3) [2025-03-17 09:57:04.944197] INFO [MDS] for_each_ls_in_tenant (mds_tenant_service.cpp:243) [117737][T1_Occam][T1][YB420A0B093E-00062FB7EFFFBFD8-0-0] [lt=17] for each ls(succ_num=1, ret=0, ret="OB_SUCCESS") [2025-03-17 09:57:04.950473] INFO [LIB] stat (utility.h:1210) [117821][T1_TenantWeakRe][T1][Y0-0000000000000000-0-0] [lt=27] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f7ac1df0050, extra_info="id:1") [2025-03-17 09:57:04.951696] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=17][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859738954}], scn_array=[cnt:1, {val:1742176624921855000, v:0}], curr_size=545, accum_size=1721, time_cost=18374) [2025-03-17 09:57:04.951790] INFO [PALF] try_advance_committed_lsn_ (log_sliding_window.cpp:1771) [3539401][T1003_LogIOCb0][T1003][Y0-0000000000000000-0-0] [lt=6] [PALF STAT COMMITTED LOG SIZE](palf_id=1, self="192.168.1.62:2882", committed size=8305) [2025-03-17 09:57:04.962999] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=5] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f79ae329470, extra_info="") [2025-03-17 09:57:04.963042] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=30] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79ae2e4050, extra_info="id:1") [2025-03-17 09:57:04.963058] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=6] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79c4a9b470, extra_info="") [2025-03-17 09:57:04.964254] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=57][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859739499}], scn_array=[cnt:1, {val:1742176624935747000, v:0}], curr_size=588, accum_size=2309, time_cost=12416) [2025-03-17 09:57:04.964487] INFO [STORAGE.TRANS] self_check (ob_tenant_weak_read_cluster_service.cpp:833) [3539253][T1004_TenantWea][T1004][Y0-0000000000000000-0-0] [lt=28] [WRS] [TENANT_WEAK_READ_SERVICE] [CLUSTER_SERVICE] [SELF_CHECK] done(ret=0, ret="OB_SUCCESS", tenant_id=1004, need_start_service=false, need_stop_service=false, need_change_leader=false, is_in_service()=true, can_update_version=true, cur_leader_epoch=1, start_service_tstamp_=1741595957992550, error_count_for_change_leader_=0, last_error_tstamp_for_change_leader_=0) [2025-03-17 09:57:04.974313] INFO [STORAGE] operator() (ob_tenant_freezer.cpp:129) [117761][T1_Occam][T1][YB420A0B093E-00062FB860254125-0-0] [lt=28] ====== tenant freeze timer task ====== [2025-03-17 09:57:04.978258] INFO [STORAGE] gc_tables_in_queue (ob_tenant_meta_mem_mgr.cpp:612) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=29] gc tables in queue: recycle 0 table(ret=0, tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl3788EEE", sizeof(T):3824, used_obj_cnt:887, free_obj_hold_cnt:105, allocator used:3817216, allocator total:4047360}, large_tablet_buffer_pool_={typeid(T).name():"N9oceanbase7storage15ObMetaObjBufferINS0_8ObTabletELl65448EEE", sizeof(T):65480, used_obj_cnt:105, free_obj_hold_cnt:890, allocator used:65263045, allocator total:65263045}, full_tablet_creator_={tiny_allocator_.used():0, tiny_allocator_.total():178560, full allocator used:1600, full allocator total:244096}, tablets_mem=69554501, tablets_mem_limit=343597380, ddl_kv_pool_={typeid(T).name():"N9oceanbase7storage7ObDDLKVE", sizeof(T):1024, used_obj_cnt:0, free_obj_hold_cnt:0, allocator used:0, allocator total:0}, memtable_pool_={typeid(T).name():"N9oceanbase8memtable10ObMemtableE", sizeof(T):1984, used_obj_cnt:23, free_obj_hold_cnt:66, allocator used:182272, allocator total:195840}, pending_cnt=0, wait_gc_count=0, tablet count=992) [2025-03-17 09:57:04.984240] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=22][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:04.986338] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=28][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162423398}], scn_array=[cnt:1, {val:1742176624663416002, v:0}], curr_size=122, accum_size=732, time_cost=23249) [2025-03-17 09:57:04.987860] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:133) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=42] ====== checkpoint timer task ====== [2025-03-17 09:57:04.987907] INFO [CLOG] get_max_applied_scn (ob_log_apply_service.cpp:745) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=23] get_max_applied_scn(scn={val:1742176624921855000, v:0}, this={ls_id_:{id:1}, role_:1, proposal_id_:1, palf_committed_end_lsn_:{lsn:4859740087}, palf_committed_end_scn_:{val:1742176624935747000, v:0}, last_check_scn_:{val:1742176624935747000, v:0}, max_applied_cb_scn_:{val:1742176624921855000, v:0}}) [2025-03-17 09:57:04.987941] INFO [CLOG] get_min_unreplayed_log_info (ob_replay_status.cpp:1024) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=30] get_min_unreplayed_log_info in skip state(lsn={lsn:0}, scn={val:1, v:0}, this={ls_id_:{id:1}, is_enabled_:true, is_submit_blocked_:false, role_:1, err_info_:{lsn_:{lsn:18446744073709551615}, scn_:{val:0, v:0}, log_type_:0, is_submit_err_:false, err_ts_:0, err_ret_:0}, ref_cnt_:2, post_barrier_lsn_:{lsn:18446744073709551615}, pending_task_count_:0, submit_log_task_:{ObReplayServiceSubmitTask:{type_:1, enqueue_ts_:1741595943665555, err_info_:{has_fatal_error_:false, fail_ts_:0, fail_cost_:0, ret_code_:0}}, next_to_submit_lsn_:{lsn:0}, next_to_submit_scn_:{val:0, v:0}, base_lsn_:{lsn:0}, base_scn_:{val:1, v:0}, iterator_:{iterator_impl:{buf_:null, next_round_pread_size:3678208, curr_read_pos:0, curr_read_buf_start_pos:0, curr_read_buf_end_pos:0, log_storage_:{start_lsn:{lsn:0}, end_lsn:{lsn:0}, read_buf:{buf_len_:0, buf_:null}, block_size:67104768, log_storage_:0x7f79aaff7a70, log_storage_:{log_tail:{lsn:4859740087}, readable_log_tail:{lsn:4859740087}, log_block_header:{magic:18754, version:1, min_lsn:{lsn:4831543296}, min_scn:{val:1742173162720054001, v:0}, curr_block_id:72, palf_id:1}, block_mgr:{log_dir:"/home/admin/oceanbase/store/sgkydb/clog/tenant_1003/1/log", dir_fd:268, min_block_id:12, max_block_id:73, curr_writable_block_id:72}, logical_block_size_:67104768, curr_block_writable_size_:38907977, block_header_serialize_buf_:0x7f79aaff8738, flashback_version:0}, storage_type:"DISK_STORAGE"}, curr_entry_is_raw_write:false, curr_entry_size:0, prev_entry_scn:{val:18446744073709551615, v:3}, curr_entry:{LogEntryHeader:{magic:0, version:0, log_size:-1, scn_:{val:18446744073709551615, v:3}, data_checksum:0, flag:0}}, init_mode_version:0, accumulate_checksum:-1, curr_entry_is_padding:0, padding_entry_size:0, padding_entry_scn:{val:18446744073709551615, v:3}}, io_ctx:{user:"REPLAY", palf_id:1, iterator_info:{allow_filling_cache:true, hot_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, cold_cache_stat:{hit_cnt:0, miss_cnt:0, cache_read_size:0, hit_ratio:"0.000000000000000000e+00"}, read_io_cnt:0, read_io_size:0, read_disk_cost_ts:0, start_lsn:{lsn:0}}}}}}) [2025-03-17 09:57:04.988692] INFO [STORAGE.TRANS] get_rec_scn (ob_trans_ctx_mgr_v4.cpp:1430) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=89] succ to get rec scn(*this={this:0x7f79c0804030, ls_id:{id:1}, tenant_id:1003, tx_ls_state_mgr:{ls_id_:{id:1}, cur_state_.state_val_:{state_:"L_WORKING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, prev_state_.state_val_:{state_:"T_APPLY_PENDING", offline_flag_:0, block_start_tx_flag_:0, block_start_normal_tx_flag_:0, block_start_readonly_flag_:0}, max_applying_start_working_ts_:{val:2, v:0}, max_applied_start_working_ts_:{val:2, v:0}}, total_tx_ctx_count:0, active_tx_count:0, ls_retain_ctx_mgr:{retain_ctx_list_.get_size():0, max_wait_ckpt_ts_:{val:18446744073709551615, v:3}, last_push_gc_task_ts_:1741595943500009, skip_remove_cnt_:0}, aggre_rec_scn:{val:18446744073709551615, v:3}, prev_aggre_rec_scn:{val:18446744073709551615, v:3}, uref:995}, aggre_rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:04.988723] INFO [STORAGE.TRANS] get_rec_scn (ob_tx_ctx_memtable.cpp:182) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=31] tx ctx memtable get rec scn(this={ObITable:{this:0x7f798c698080, key:{tablet_id:{id:49401}, column_group_idx:0, table_type:"TX_CTX_MEMTABLE", scn_range:{start_scn:{val:1, v:0}, end_scn:{val:1741716004436780, v:0}}}, ref_cnt:2, upper_trans_version:-4007, timestamp:0}, this:0x7f798c698080, snapshot_version:{val:1741716004436780, v:0}, ls_id:{id:1}, is_frozen:false}, rec_scn={val:4611686018427387903, v:0}) [2025-03-17 09:57:04.988811] INFO [MDS] get_rec_scn (mds_table_mgr.cpp:265) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=22] [FREEZE]get rec_scn from MdsTableMgr(ret=0, ret="OB_SUCCESS", min_rec_scn={val:1742157237104149001, v:0}, order_flusher_for_one={high_priority_flusher:size:1, ordered_tablets:{{tablet_id:{id:226}, rec_scn:{val:1742157237104149001, v:0}}, }, *this={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}) [2025-03-17 09:57:04.988865] INFO [STORAGE.TRANS] get_rec_scn (ob_ls_tx_service.cpp:637) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=51] [CHECKPOINT] ObLSTxService::get_rec_scn(common_checkpoint_type="MDS_TABLE_TYPE", common_checkpoints_[min_rec_scn_common_checkpoint_type_index]={this:0x7f79a3daf150, is_inited:true, freezing_scn:{val:1742059159250522001, v:0}, ls:{running_state:{state:1}, ls_meta:{tenant_id:1003, ls_id:{id:1}, ls_persistent_state:{state:1}, clog_checkpoint_scn:{val:1742157237104149001, v:0}, clog_base_lsn:{lsn:4697333760}, rebuild_seq:0, migration_status:0, gc_state_:1, offline_scn_:{val:18446744073709551615, v:3}, restore_status:{status:0}, replayable_point:{val:18446744073709551615, v:3}, tablet_change_checkpoint_scn:{val:1741942655064529000, v:0}, all_id_meta:{id_meta:[{limited_id:1742176636190689000, latest_log_ts:{val:1742176619532124001, v:0}}, {limited_id:11000001, latest_log_ts:{val:1742160213403604001, v:0}}, {limited_id:70000001, latest_log_ts:{val:1742161213638166001, v:0}}]}, transfer_scn:{val:0, v:0}, rebuild_info:{status:{status:0}, type:{type:0}, tablet_id_array:{id_array:[]}, src:[type = 0], [address = ]}, transfer_meta_info:{src_ls:{id:0}, src_scn:{val:1, v:0}, trans_status:0, tablet_id_array:{id_array:[]}, data_version:1(0, 0, 1)}, store_format:{store_type:1, store_type_str:"NORMAL"}}, switch_epoch:1, log_handler:{role:1, proposal_id:1, palf_env_:0x7f79a49e4030, is_in_stop_state_:false, is_inited_:true, id_:1}, restore_handler:{is_inited:true, is_in_stop_state:false, id:1, proposal_id:9223372036854775807, role:2, parent:null, context:{issue_task_num:0, issue_version:-1, last_fetch_ts:-1, max_submit_lsn:{lsn:18446744073709551615}, max_fetch_lsn:{lsn:18446744073709551615}, max_fetch_scn:{val:18446744073709551615, v:3}, error_context:{ret_code:0, trace_id:Y0-0000000000000000-0-0, error_type:0, err_lsn:{lsn:18446744073709551615}}, task_count:0}, restore_context:{seek_done:false, lsn:{lsn:18446744073709551615}}}, is_inited:true, tablet_gc_handler:{tablet_persist_trigger:0, is_inited:true}, startup_transfer_info:{ls_id:{id:0}, transfer_start_scn:{val:18446744073709551615, v:3}}, need_delay_resource_recycle:false}}, min_rec_scn={val:1742157237104149001, v:0}, ls_id_={id:1}) [2025-03-17 09:57:04.988903] INFO [STORAGE] get_rec_scn (ob_ls_ddl_log_handler.cpp:501) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=34] [CHECKPOINT] ObLSDDLLogHandler::get_rec_scn(ret=0, ls_id={id:1}, barrier_tablet_id={id:0}, rec_scn={val:4611686018427387903, v:0}, last_rec_scn={val:1, v:0}) [2025-03-17 09:57:04.988914] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=10] get rec log scn(service_type_=0, rec_log_ts={val:1742176619532124001, v:0}) [2025-03-17 09:57:04.988922] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=7] get rec log scn(service_type_=1, rec_log_ts={val:1742160213403604001, v:0}) [2025-03-17 09:57:04.988932] INFO [STORAGE.TRANS] get_rec_scn (ob_id_service.cpp:308) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=8] get rec log scn(service_type_=2, rec_log_ts={val:1742161213638166001, v:0}) [2025-03-17 09:57:04.988945] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=9] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:04.988964] INFO [STORAGE.TABLELOCK] get_rec_scn (ob_lock_memtable.cpp:1220) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=16] rec_scn of ObLockMemtable is (rec_scn_={val:4611686018427387903, v:0}, flushed_scn_={val:0, v:0}, pre_rec_scn_={val:4611686018427387903, v:0}, freeze_scn_={val:0, v:0}, max_committed_scn_={val:18446744073709551615, v:3}, is_frozen_=false, ls_id_={id:1}) [2025-03-17 09:57:04.988979] INFO [STORAGE] update_clog_checkpoint (ob_checkpoint_executor.cpp:189) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=13] [CHECKPOINT] clog checkpoint no change(checkpoint_scn={val:1742157237104149001, v:0}, checkpoint_scn_in_ls_meta={val:1742157237104149001, v:0}, ls_id={id:1}, service_type="TRANS_SERVICE") [2025-03-17 09:57:04.988997] WDIAG [PALF] set_base_lsn (palf_handle_impl.cpp:1664) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=13][errcode=0] no need to set new base lsn, curr base lsn is greater than or equal to new base lsn(this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, log_snapshot_meta={version:2, base_lsn:{lsn:4697333760}, prev_log_info:{log_id:0, lsn:{lsn:0}, scn:{val:1, v:0}, log_proposal_id:9223372036854775807, accum_checksum:-1}, prev_log_tail_lsn:{lsn:0}}, new_base_lsn={lsn:4697333760}, lsn={lsn:4697333760}) [2025-03-17 09:57:04.989013] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:190) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=15] [CHECKPOINT] advance palf base lsn successfully(checkpoint_lsn={lsn:4697333760}, ls->get_ls_id()={id:1}) [2025-03-17 09:57:04.989022] INFO [STORAGE] runTimerTask (ob_checkpoint_service.cpp:197) [3539393][T1003_TimerWK1][T1003][Y0-0000000000000000-0-0] [lt=7] succeed to update_clog_checkpoint(ret=0, ls_cnt=1) [2025-03-17 09:57:05.022757] INFO [DETECT] record_summary_info_and_logout_when_necessary_ (ob_lcl_batch_sender_thread.cpp:203) [117815][T1_LCLSender][T1][Y0-0000000000000000-0-0] [lt=39] ObLCLBatchSenderThread periodic report summary info(duty_ratio_percentage=0, total_constructed_detector=0, total_destructed_detector=0, total_alived_detector=0, _lcl_op_interval=30000, lcl_msg_map_.count()=0, *this={this:0x7f7b0abbc2b0, is_inited:true, is_running:true, total_record_time:5010000, over_night_times:0}) [2025-03-17 09:57:05.023008] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=26] sql audit evict task end(request_manager_->get_tenant_id()=1004, evict_high_mem_level=371085173, evict_high_size_level=589824, evict_batch_count=0, elapse_time=1, size_used=131072, mem_used=241250304) [2025-03-17 09:57:05.023077] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539207][T1004_TimerWK1][T1004][Y0-0000000000000000-0-0] [lt=34] Construct Queue Num(construct_num=0, get_push_idx()=244, get_cur_idx()=236, get_pop_idx()=234) [2025-03-17 09:57:05.030323] INFO [COMMON] compute_tenant_wash_size (ob_kvcache_store.cpp:1136) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=43] Wash compute wash size(is_wash_valid=false, sys_total_wash_size=-40428850380, global_cache_size=9209479168, tenant_max_wash_size=0, tenant_min_wash_size=0, tenant_ids_=[cnt:6, 500, 508, 509, 1, 1003, 1004]) [2025-03-17 09:57:05.050789] WDIAG [SQL.ENG] get_next_left_row (ob_nested_loop_join_vec_op.cpp:167) [117882][T1_L5_G0][T1][YB420A0B093F-00062FB7B25CA58A-0-0] [lt=33][errcode=-4008] failed to get next left row from friver iterator(ret=-4008) [2025-03-17 09:57:05.063105] INFO [LIB] stat (utility.h:1210) [3539261][T1004_TxLoopWor][T1004][Y0-0000000000000000-0-0] [lt=9] [PALF STAT APPEND COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=0, this=0x7f79e6d1e050, extra_info="id:1001") [2025-03-17 09:57:05.064719] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [117781][T1_TimerWK0][T1][Y0-0000000000000000-0-0] [lt=50] sql audit evict task end(request_manager_->get_tenant_id()=1, evict_high_mem_level=154618822, evict_high_size_level=530841, evict_batch_count=0, elapse_time=0, size_used=65536, mem_used=126864384) [2025-03-17 09:57:05.064772] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [117782][T1_TimerWK1][T1][Y0-0000000000000000-0-0] [lt=29] Construct Queue Num(construct_num=0, get_push_idx()=1364, get_cur_idx()=1356, get_pop_idx()=1355) [2025-03-17 09:57:05.077547] INFO [COMMON] replace_map (ob_kv_storecache.cpp:776) [117407][TimerWK0][T0][Y0-0000000000000000-0-0] [lt=34] replace map num details(ret=0, replace_node_count=0, map_once_replace_num_=100000, map_replace_skip_count_=6) [2025-03-17 09:57:05.080361] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:775) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=24] success to update min reserved snapshot(reserved_snapshot=1742174815679197000, duration=1800, snapshot_gc_ts=1742176615679197000) [2025-03-17 09:57:05.081190] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539217][T1004_IOWorker][T1004][Y0-0000000000000000-0-0] [lt=41][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1001, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4162423520}], scn_array=[cnt:1, {val:1742176624663416003, v:0}], curr_size=122, accum_size=854, time_cost=18077) [2025-03-17 09:57:05.081790] INFO [LIB] stat (utility.h:1210) [117829][T1_TxLoopWorker][T1][Y0-0000000000000000-0-0] [lt=24] [PALF STAT PUSH LOG TO FOLLOWERS RPC POST COST TIME](cur_stat_count=10, stat_interval=1000000, avg (count/cost)=1, this=0x7f7adde5d470, extra_info="") [2025-03-17 09:57:05.084314] WDIAG [STORAGE.TRANS] run1 (ob_standby_timestamp_service.cpp:147) [3539252][T1004_STSWorker][T1004][Y0-0000000000000000-0-0] [lt=27][errcode=-4076] query and update last id fail(ret=-4076, ret="OB_NEED_WAIT") [2025-03-17 09:57:05.086880] INFO [PALF] sliding_cb (log_sliding_window.cpp:2291) [3539215][T1004_LogIOCb0][T1004][Y0-0000000000000000-0-0] [lt=27] [PALF STAT LOG TASK TIME](palf_id=1001, self="192.168.1.62:2882", total_slide_log_cnt=11, avg_log_gen_to_freeze_time=4, avg_log_gen_to_submit_time=7, avg_log_submit_to_flush_time=18002, avg_log_submit_to_slide_time=18019) [2025-03-17 09:57:05.086945] INFO [PALF] read (log_cache.cpp:103) [3539221][T1004_LogShared][T1004][Y0-0000000000000000-0-0] [lt=25] [PALF STAT HOT CACHE HIT RATE](palf_id=1001, read_size=1332, hit_cnt=11, read_cnt=11, hit rate=1.000000000000000000e+00) [2025-03-17 09:57:05.086964] INFO [STORAGE] try_update_reserved_snapshot (ob_tenant_freeze_info_mgr.cpp:802) [3539206][T1004_TimerWK0][T1004][Y0-0000000000000000-0-0] [lt=29] update reserved snapshot finished(cost_ts=6604, reserved_snapshot=1742174815679197000) [2025-03-17 09:57:05.100023] INFO [PALF] handle_next_submit_log_ (log_sliding_window.cpp:1093) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=9] [PALF STAT GROUP LOG INFO](palf_id=1, self="192.168.1.62:2882", role="LEADER", total_group_log_cnt=13, avg_log_batch_cnt=1, total_group_log_size=8305, avg_group_log_size=638) [2025-03-17 09:57:05.100064] INFO [LIB] stat (utility.h:1210) [3539427][T1003_TenantWea][T1003][Y0-0000000000000000-0-0] [lt=36] [PALF STAT APPEND COST TIME](cur_stat_count=13, stat_interval=1000000, avg (count/cost)=0, this=0x7f79a3db8050, extra_info="id:1") [2025-03-17 09:57:05.119196] INFO [STORAGE.BLKMGR] runTimerTask (ob_block_manager.cpp:1656) [117409][TimerWK2][T0][Y0-0000000000000000-0-0] [lt=57] skip inspect bad block(last_check_time=1742008232806107, last_macro_idx=-1) [2025-03-17 09:57:05.127644] WDIAG [PALF] inner_append_log (palf_handle_impl.cpp:2228) [3539402][T1003_IOWorker][T1003][Y0-0000000000000000-0-0] [lt=43][errcode=-4389] write log cost too much time(ret=-4389, this={palf_id:1, self:"192.168.1.62:2882", has_set_deleted:false}, lsn_array=[cnt:1, {lsn:4859740087}], scn_array=[cnt:1, {val:1742176625099416000, v:0}], curr_size=588, accum_size=2897, time_cost=27606) [2025-03-17 09:57:05.127873] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:41) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=28] ====== [emptytablet] empty shell timer task ======(GC_EMPTY_TABLET_SHELL_INTERVAL=5000000) [2025-03-17 09:57:05.127899] INFO [STORAGE] runTimerTask (ob_empty_shell_task.cpp:107) [3539209][T1004_TimerWK3][T1004][Y0-0000000000000000-0-0] [lt=19] [emptytablet] succeed to change tablet to empty shell(ret=0, ret="OB_SUCCESS", ls_cnt=2, times=11666) [2025-03-17 09:57:05.131202] INFO [SERVER] runTimerTask (ob_eliminate_task.cpp:225) [3539394][T1003_TimerWK2][T1003][Y0-0000000000000000-0-0] [lt=41] sql audit evict task end(request_manager_->get_tenant_id()=1003, evict_high_mem_level=25769803, evict_high_size_level=471859, evict_batch_count=0, elapse_time=1, size_used=0, mem_used=20797440) [2025-03-17 09:57:05.131238] INFO [SERVER] prepare_alloc_queue (ob_dl_queue.cpp:52) [3539392][T1003_TimerWK0][T1003][Y0-0000000000000000-0-0] [lt=29] Construct Queue Num(construct_num=0, get_push_idx()=521, get_cur_idx()=513, get_pop_idx()=513) [2025-03-17 09:57:05.132492] INFO [COMMON] print_sender_status (ob_io_manager.cpp:1371) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=7] [IO STATUS SENDER](io_status=req_in_sender, total=0, total_local_read=0, total_local_write=0, total_remote_read=0, total_remote_write=0, cnt=0 0 0 0 0 0 0 0 , local_read=0 0 0 0 0 0 0 0 , local_write=0 0 0 0 0 0 0 0 , remote_read=0 0 0 0 0 0 0 0 , remote_write=0 0 0 0 0 0 0 0 ) [2025-03-17 09:57:05.132551] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=29] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:MICRO_BLOCK_CACHE_IO, mode:LOCAL READ, cur_req:0, hold_mem:0 [FAILED]: fail_size:4096, fail_iops:6, fail_bw:24532, [delay/us]:prepare:1, schedule:24, submit:1, rt:392, total:419, [SUCC]: size:0, iops:0, bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0) [2025-03-17 09:57:05.132575] INFO [COMMON] print_io_status (ob_io_manager.cpp:2700) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=12] [IO STATUS GROUP SYS](tenant_id=1, io_status=sys_group_name:CLOG_WRITE_IO, mode:LOCAL WRITE, cur_req:0, hold_mem:0 [FAILED]: fail_size:0, fail_iops:0, fail_bw:0, [delay/us]:prepare:0, schedule:0, submit:0, rt:0, total:0, [SUCC]: size:4505, iops:10, bw:44977, [delay/us]:prepare:5, schedule:18, submit:24, rt:10648, total:10697) [2025-03-17 09:57:05.132589] INFO [COMMON] print_io_status (ob_io_manager.cpp:2730) [117627][IO_TUNING0][T0][Y0-0000000000000000-0-0] [lt=9] [IO STATUS TENANT](tenant_id=1, ref_cnt=2, io_config={group_configs_cnt:6, memory_limit_:5368709120, callback_thread_count_:0, unit_config_:{min_iops:9223372036854775807, max_iops:9223372036854775807, weight:4, max_net_bandwidth:9223372036854775807, net_bandwidth_weight:4}, enable_io_tracer:false, object_storage_io_timeout_ms:20000, group_configs:[group_id = 0:{mode:0, deleted:false, cleared:false, min_percent:0, max_percent:100, weight_percent:100}, group_id = 0:{mode:1, deleted:fal